Adds a `parallel` flag to `deno serve`. When present, we spawn multiple
workers to parallelize serving requests.
```bash
deno serve --parallel main.ts
```
Currently on linux we use `SO_REUSEPORT` and rely on the fact that the
kernel will distribute connections in a round-robin manner.
On mac and windows, we sort of emulate this by cloning the underlying
file descriptor and passing a handle to each worker. The connections
will not be guaranteed to be fairly distributed (and in practice almost
certainly won't be), but the distribution is still spread enough to
provide a significant performance increase.
---
(Run on an Macbook Pro with an M3 Max, serving `deno.com`
baseline::
```
❯ wrk -d 30s -c 125 --latency http://127.0.0.1:8000
Running 30s test @ http://127.0.0.1:8000
2 threads and 125 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 239.78ms 13.56ms 330.54ms 79.12%
Req/Sec 258.58 35.56 360.00 70.64%
Latency Distribution
50% 236.72ms
75% 248.46ms
90% 256.84ms
99% 268.23ms
15458 requests in 30.02s, 2.47GB read
Requests/sec: 514.89
Transfer/sec: 84.33MB
```
this PR (`with --parallel` flag)
```
❯ wrk -d 30s -c 125 --latency http://127.0.0.1:8000
Running 30s test @ http://127.0.0.1:8000
2 threads and 125 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 117.40ms 142.84ms 590.45ms 79.07%
Req/Sec 1.33k 175.19 1.77k 69.00%
Latency Distribution
50% 22.34ms
75% 223.67ms
90% 357.32ms
99% 460.50ms
79636 requests in 30.07s, 12.74GB read
Requests/sec: 2647.96
Transfer/sec: 433.71MB
```
I noticed
[`set_response_body`](ce42f82b5a/ext/http/service.rs (L439-L443))
was unexpectedly hot in profiles, with most of the time being spent in
`memmove`.
It turns out that `ResponseBytesInner` was _massive_ (5624 bytes), so
every time we moved a `ResponseBytesInner` (for instance in
`set_response_body`) we were doing a >5kb memmove, which adds up pretty
quickly.
This PR boxes the two larger variants (the compression streams),
shrinking `ResponseBytesInner` to a reasonable 48 bytes.
---
Benchmarked with a simple hello world server:
```ts
// hello-server.ts
Deno.serve((_req) => {
return new Response("Hello world");
});
// run with `deno run -A hello-server.ts`
// in separate terminal `wrk -d 10s http://127.0.0.1:8000`
```
Main:
```
Running 10s test @ http://127.0.0.1:8000/
2 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 53.39us 9.53us 0.98ms 92.78%
Req/Sec 86.57k 3.56k 91.58k 91.09%
1739319 requests in 10.10s, 248.81MB read
Requests/sec: 172220.92
Transfer/sec: 24.64MB
```
This PR:
```
Running 10s test @ http://127.0.0.1:8000/
2 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 45.44us 8.49us 0.91ms 90.04%
Req/Sec 100.65k 2.26k 102.65k 96.53%
2022296 requests in 10.10s, 289.29MB read
Requests/sec: 200226.20
Transfer/sec: 28.64MB
```
So a nice ~15% bump. (With response body compression, the gain is ~10%
for gzip and neutral for brotli)
This is the release commit being forwarded back to main for 1.45.3
---------
Co-authored-by: bartlomieju <bartlomieju@users.noreply.github.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This is the release commit being forwarded back to main for 1.44.3
Co-authored-by: bartlomieju <bartlomieju@users.noreply.github.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This is the release commit being forwarded back to main for 1.44.1
Co-authored-by: devsnek <devsnek@users.noreply.github.com>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
This commit changes `gzip` compression in `Deno.serve` API to flush data
after each write. There's a slight performance regression, but provided
test shows a scenario that was not possible before.
---------
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
This is a primordialization effort to improve resistance against users
tampering with the global `Object` prototype.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
1. Generally we should prefer to use the `log` crate.
2. I very often accidentally commit `eprintln`s.
When we should use `println` or `eprintln`, it's not too bad to be a bit
more verbose and ignore the lint rule.
**THIS PR HAS GIT CONFLICTS THAT MUST BE RESOLVED**
This is the release commit being forwarded back to main for 1.43.2
Please ensure:
- [x] Everything looks ok in the PR
- [x] The release has been published
To make edits to this PR:
```shell
git fetch upstream forward_v1.43.2 && git checkout -b forward_v1.43.2 upstream/forward_v1.43.2
```
Don't need this PR? Close it.
cc @nathanwhit
Co-authored-by: nathanwhit <nathanwhit@users.noreply.github.com>
Co-authored-by: Nathan Whitaker <nathan@deno.com>
By default, `deno serve` will assign port 8000 (like `Deno.serve`).
Users may choose a different port using `--port`.
`deno serve /tmp/file.ts`
`server.ts`:
```ts
export default {
fetch(req) {
return new Response("hello world!\n");
},
};
```
When the response has been successfully send, we abort the
`Request.signal` property to indicate that all resources associated with
this transaction may be torn down.
Landing part of https://github.com/denoland/deno/pull/21903
This will allow us to more easily refactor `serveHttp` to live on top of
`serve` by splitting the websocket code out. There's probably a lot more
we could do here but this helps.
Adds an `addr` field to `HttpServer` to simplify the pattern
`Deno.serve({ onListen({ port } => listenPort = port })`. This becomes:
`const server = Deno.serve({}); port = server.addr.port`.
Changes:
- Refactors `serve` overloads to split TLS out (in preparation for
landing a place for the TLS SNI information)
- Adds an `addr` field to `HttpServer` that matches the `addr` field of
the corresponding `Deno.Listener`s.
Changes:
- Implements a TCP socket listener that will allow for round-robin
load-balancing in-process.
- Cleans up the raw networking code to make it easier to work with.
