Gets us closer to solving #20707.
Rewrites the `TestEventSender`:
- Allow for explicit creation of multiple streams. This will allow for
one-std{out,err}-per-worker
- All test events are received along with a worker ID, allowing for
eventual, proper parallel threading of test events.
In theory this should open up proper interleaving of test output,
however that is left for a future PR.
I had some plans for a better performing synchronization primitive, but
the inter-thread communication is tricky. This does, however, speed up
the processing of large numbers of tests 15-25% (possibly even more on
100,000+).
Before
```
ok | 1000 passed | 0 failed (32ms)
ok | 10000 passed | 0 failed (276ms)
```
After
```
ok | 1000 passed | 0 failed (25ms)
ok | 10000 passed | 0 failed (230ms)
```
Partially supersedes #19016.
This migrates `spawn` and `spawn_blocking` to `deno_core`, and removes
the requirement for `spawn` tasks to be `Send` given our single-threaded
executor.
While we don't need to technically do anything w/`spawn_blocking`, this
allows us to have a single `JoinHandle` type that works for both cases,
and allows us to more easily experiment with alternative
`spawn_blocking` implementations that do not require tokio (ie: rayon).
Async ops (+~35%):
Before:
```
time 1310 ms rate 763358
time 1267 ms rate 789265
time 1259 ms rate 794281
time 1266 ms rate 789889
```
After:
```
time 956 ms rate 1046025
time 954 ms rate 1048218
time 924 ms rate 1082251
time 920 ms rate 1086956
```
HTTP serve (+~4.4%):
Before:
```
Running 10s test @ http://localhost:4500
2 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 68.78us 19.77us 1.43ms 86.84%
Req/Sec 68.78k 5.00k 73.84k 91.58%
1381833 requests in 10.10s, 167.36MB read
Requests/sec: 136823.29
Transfer/sec: 16.57MB
```
After:
```
Running 10s test @ http://localhost:4500
2 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 63.12us 17.43us 1.11ms 85.13%
Req/Sec 71.82k 3.71k 77.02k 79.21%
1443195 requests in 10.10s, 174.79MB read
Requests/sec: 142921.99
Transfer/sec: 17.31MB
```
Suggested-By: alice@ryhl.io
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
1. Boxed `File` and `FileSystem` to allow more easily passing this
through the CLI code (as shown within this pr).
2. `StdFileResource` is now `FileResource`. `FileResource` now contains
an `Rc<dyn File>`.
This removes the tty stuff that's hanging on the file resources and
instead stores them in a separate `TtyModeStore`. Although this will
cause the tty store items to not be removed when the resource is
removed, I think this is ok to do because there will be a small number
of resources this is every done with and usually those resources won't
ever be closed.
2 fixes related to sync fs:
* update the 2 sync methods on `Resource` trait to take `Rc<Self>`
(consistent with other methods)
* fix a bug in `StdFileResource::with_inner_and_metadata`, which
currently can trigger a panic if a sync method is called on a file with
a pending async operation. This could happen in the code path where
`File::try_clone`
[fails](39ece1fe0d/ext/io/lib.rs (L485-L489)).
This commit abstracts out the specifics of the underlying system calls
FS operations behind a new `FileSystem` and `File` trait in the
`ext/fs` extension.
This allows other embedders to re-use ext/fs, but substituting in a
different FS backend.
This is likely not the final form of these traits. Eventually they will
be entirely `deno_core::Resource` agnostic, and will live in a seperate
crate.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit adds op_read_sync and op_write_sync to core. These ops are
similar to op_read and op_write, but they are synchronous. Just like the
async ops, they operate on generic `deno_core::Resource` objects. These
now have new `read_byob_sync` and `write_sync` methods, with default
implementations throwing "NotSupported" errors, just like the async
counterparts.
There are no `write_all` or `read` equivalents, because the
optimizations they unlock are not useful in synchronous contexts.
Follow-up to #18210:
* we are passing the generated `cfg` object into the state function
rather than passing individual config fields
* reduce cloning dramatically by making the state_fn `FnOnce`
* `take` for `ExtensionBuilder` to avoid more unnecessary copies
* renamed `config` to `options`
This implements two macros to simplify extension registration and centralize a lot of the boilerplate as a base for future improvements:
* `deno_core::ops!` registers a block of `#[op]`s, optionally with type
parameters, useful for places where we share lists of ops
* `deno_core::extension!` is used to register an extension, and creates
two methods that can be used at runtime/snapshot generation time:
`init_ops` and `init_ops_and_esm`.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit splits "<ext_name>::init" functions into "init_ops" and
"init_ops_and_esm". That way we don't have to construct list of
ESM sources on each startup if we're running with a snapshot.
In a follow up commit "deno_core" will be changed to not have a split
between "extensions" and "extensions_with_js" - it will be embedders'
responsibility to pass appropriately configured extensions.
Prerequisite for https://github.com/denoland/deno/pull/18080
There's no point for this API to expect result. If something fails it should
result in a panic during build time to signal to embedder that setup is
wrong.
