Currently all async ops are polled lazily, which means that op
initialization code is postponed until control is yielded to the event
loop. This has some weird consequences, e.g.
```js
let listener = Deno.listen(...);
let conn_promise = listener.accept();
listener.close();
// `BadResource` is thrown. A reasonable error would be `Interrupted`.
let conn = await conn_promise;
```
JavaScript promises are expected to be eagerly evaluated. This patch
makes ops actually do that.
Decouple JsRuntime::sync_ops_cache() from the availability of the Deno.* namespace in the global scope
This avoids crashes when calling sync_ops_cache() on a bootstrapped WebWorker who has dropped its Deno.* namespace
It's also just cleaner and more robust ...
This commit fixes a problem where loading and executing multiple
modules leads to all of the having "import.meta.main" set to true.
Following Rust APIs were deprecated:
- deno_core::JsRuntime::load_module
- deno_runtime::Worker::execute_module
- deno_runtime::WebWorker::execute_module
Following Rust APIs were added:
- deno_core::JsRuntime::load_main_module
- deno_core::JsRuntime::load_side_module
- deno_runtime::Worker::execute_main_module
- deno_runtime::Worker::execute_side_module
- deno_runtime::WebWorker::execute_main_module
Trying to load multiple "main" modules into the runtime now results in an
error. If user needs to load additional "non-main" modules they should use
APIs for "side" module.
Oneshot is more appropriate because mod_evaluate() only sends a single
value.
It also makes it easier to use it correctly. As an embedder, I wasn't
sure if I'm expected to drain the channel or not.
This commit changes return type of JsRuntime::execute_script to include
v8::Value returned from evaluation.
When embedding deno_core it is sometimes useful to be able to inspect
script evaluation value without the hoops of adding ops to store the
value on the OpState.
v8::Global<v8::Value> is used so consumers don't have to pass
scope themselves.
The WebAssembly streaming APIs used to be enabled, but used to take
buffer sources as their first argument (see #6154 and #7259). This
change re-enables them, requiring a Promise<Response> instead, as well as
enabling asynchronous compilation of WebAssembly modules.
This commit introduces primordials to deno_core. Primordials are a
frozen set of all intrinsic objects in the runtime. They are not
vulnerable to prototype pollution.
This commit changes implementation of module loading in "deno_core"
to track all currently fetched modules across all existing module loads.
In effect a bug that caused concurrent dynamic imports referencing the
same module to fail is fixed.
Starting with V8 9.1, top-level-await is always enabled by default.
See https://v8.dev/blog/v8-release-91 for the release notes.
- Remove the now redundant v8 flag.
- Clarify doc comment and add link to the feature explainer.
This commit renames "JsRuntime::execute" to "JsRuntime::execute_script". Additionally
same renames were applied to methods on "deno_runtime::Worker" and
"deno_runtime::WebWorker".
A new macro was added to "deno_core" called "located_script_name" which
returns the name of Rust file alongside line no and col no of that call site.
This macro is useful in combination with "JsRuntime::execute_script"
and allows to provide accurate place where "one-off" JavaScript scripts
are executed for internal runtime functions.
Co-authored-by: Nayeem Rahman <nayeemrmn99@gmail.com>
This commit changes module loading implementation in "deno_core"
to call "ModuleLoader::prepare" hook only once per entry point.
This is done to avoid multiple type checking of the same code
in case of duplicated dynamic imports.
Relevant code in "cli/module_graph.rs" was updated as well.
This speeds up incremental rebuild when only touching JS files by 13-15%
Rebuild time after `touch 01_broadcast_channel.js`:
main: run 1 49.18s, run 2 50.34s
this: run 1 43.12s, run 2 43.19s
This commit moves implementation of "JsRuntimeInspector" to "deno_core" crate.
To achieve that following changes were made:
* "Worker" and "WebWorker" no longer own instance of "JsRuntimeInspector",
instead it is now owned by "deno_core::JsRuntime".
* Consequently polling of inspector is no longer done in "Worker"/"WebWorker",
instead it's done in "deno_core::JsRuntime::poll_event_loop".
* "deno_core::JsRuntime::poll_event_loop" and "deno_core::JsRuntime::run_event_loop",
now accept "wait_for_inspector" boolean that tells if event loop should still be
"pending" if there are active inspector sessions - this change fixes the problem
that inspector disconnects from the frontend and process exits once the code has
stopped executing.
This commit moves bulk of the logic related to module loading
from "JsRuntime" to "ModuleMap".
Next steps are to rewrite the actual loading logic (represented by
"RecursiveModuleLoad") to be a part of "ModuleMap" as well --
that way we will be able to track multiple module loads from within
the map which should help me solve the problem of concurrent
loads (since all info about currently loading/loaded modules will
be contained in the ModuleMap, so we'll be able to know if actually
all required modules have been loaded).
This ensures that provided extensions are all correctly setup and ready to use once the JsRuntime constructor returns
Note: this will also initialize ops for to-be-snapshotted runtimes
Extensions allow declarative extensions to "JsRuntime" (ops, state, JS or middleware).
This allows for:
- `op_crates` to be plug-and-play & self-contained, reducing complexity leaked to consumers
- op middleware (like metrics_op) to be opt-in and for new middleware (unstable, tracing,...)
- `MainWorker` and `WebWorker` to be composable, allowing users to extend workers with their ops whilst benefiting from the other infrastructure (inspector, etc...)
In short extensions improve deno's modularity, reducing complexity and leaky abstractions for embedders and the internal codebase.
General cleanup of module loading code, tried to reduce indentation in various methods
on "JsRuntime" to improve readability.
Added "JsRuntime::handle_scope" helper function, which returns a "v8::HandleScope".
This was done to reduce a code pattern that happens all over the "deno_core".
Additionally if event loop hangs during loading of dynamic modules a list of
currently pending dynamic imports is printed.
Even if bootstrapping the JS runtime is low level, it's an abstraction leak of
core to require users to call `Deno.core.ops()` in JS space.
So instead we're introducing a `JsRuntime::sync_ops_cache()` method,
once we have runtime extensions a new runtime will ensure the ops
cache is setup (for the provided extensions) and then loading/unloading
plugins should be the only operations that require op cache syncs
`init()` was previously needed to init the shared queue, but now that it's
gone `init()` only registers the async msg handler which is snapshot
safe and constant since the op layer refactor.
- Improves op performance.
- Handle op-metadata (errors, promise IDs) explicitly in the op-layer vs
per op-encoding (aka: out-of-payload).
- Remove shared queue & custom "asyncHandlers", all async values are
returned in batches via js_recv_cb.
- The op-layer should be thought of as simple function calls with little
indirection or translation besides the conceptually straightforward
serde_v8 bijections.
- Preserve concepts of json/bin/min as semantic groups of their
inputs/outputs instead of their op-encoding strategy, preserving these
groups will also facilitate partial transitions over to v8 Fast API for the
"min" and "bin" groups
In case JavaScript throws an unhandled error, part of the "shared_queue" could
be still unprocessed.
If this is the case; throw the js runtime error instead of asserting on the
queue size not being 0.
This commit rewrites implementation of "JsRuntime::mod_evaluate".
Event loop is no longer polled automatically and users must manually
drive event loop forward after calling "mod_evaluate".
Co-authored-by: Nayeem Rahman <nayeemrmn99@gmail.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit rewrites "JsRuntime::poll" function to fix a corner case that
might caused "overflown_response" to be overwritten by other overflown response.
The logic has been changed to allow returning multiple overflown response
alongside responses from shared queue.
This commit adds two new "Deno.core" APIs:
- "Deno.core.serialize"
- "Deno.core.deserialize"
These APIs are used to provide structured cloning of values
and will be used for further web worker implementation.
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit rewrites initialisation of the "shared queue" and
in effect prevents from double execution of "core/core.js" and
"core/error.js".
Previously both of these files were executed every time a "JsRuntime"
was created. That lead to a situation where one copy of each script
was included in the snapshot and then another copy would be
executed after loading the snapshot.
Effectively "JsRuntime::shared_init" was removed; instead execution
of those scripts and actual initialisation of shared queue
was split into two helper functions: "JsRuntime::js_init" and
"JsRuntime::share_queue_init".
Additionally stale TODO comments were removed.
Implementors of `deno_core::JsRuntime` might want to do additional actions
during each turn of event loop, eg. `deno_runtime::Worker` polls inspector,
`deno_runtime::WebWorker` receives/dispatches messages from/to worker host.
Previously `JsRuntime::mod_evaluate` was implemented in such fashion that it
only polled `JsRuntime`'s event loop. This behavior turned out to be wrong
in the example of `WebWorker` which couldn't receive/dispatch messages because
its implementation of event loop was never called.
This commit rewrites "mod_evaluate" to return a handle to receiver that resolves
when module's promise resolves. It is now implementors responsibility to poll
event loop after calling `mod_evaluate`.
This commit does major refactor of "Worker" and "WebWorker",
in order to decouple them from "ProgramState" and "Flags".
The main points of interest are "create_main_worker()" and
"create_web_worker_callback()" functions which are responsible
for creating "Worker" and "WebWorker" in CLI context.
As a result it is now possible to factor out common "runtime"
functionality into a separate crate.
This commit adds "Deno.core.createPrepareStackTrace". This function
was moved from "cli/rt/40_error_stack.js" to unify handling of stack frames in core
(before this PR there was implicit dependency on logic in "core/error.rs::JsError").
Unfortunately formatting logic must still be duplicated in "cli/error.js::PrettyJsError"
to provide coloring, but currently there's no solution to this problem.
"createPrepareStackTrace" can accept a single argument; a function that takes
a location and provides source mapped location back.
This commit disables source mapping of errors
for standalone binaries. Since applying source
maps relies on using file fetcher infrastructure
it's not feasible to use it for standalone binaries
that are not supposed to use that infrastructure.
Fixes panic occurring in worker when "self.close()" is called
at the top level, ie. worker shuts down while
module evaluation promise hasn't yet resolved.
This commit refactors "deno_core::Modules" structure to not depend on
"get_identity_hash" function to identify modules, but instead use default
hash implementation.
Make JSRuntime::new() accept a custom v8::CreateParams object to tune
the v8::Isolate it creates.
Subsumes the functionality of HeapLimits, which I therefore removed.
This commit changes implementation of top-level-await in "deno_core".
Previously promise returned from module evaluation was not awaited,
leading to out-of-order execution of modules that have TLA. It's been
fixed by changing "JsRuntime::mod_evaluate" to be an async function
that resolves when the promise returned from module evaluation also
resolves. When waiting for promise resolution event loop is polled
repeatedly, until there are no more dynamic imports or pending
ops.
* Revert "refactor: Worker is not a Future (#7895)"
This reverts commit f4357f0ff9.
* Revert "refactor(core): JsRuntime is not a Future (#7855)"
This reverts commit d8879feb8c.
* Revert "fix(core): module execution with top level await (#7672)"
This reverts commit c7c7677825.
This commit rewrites deno_core::JsRuntime to not implement Future
trait.
Instead there are two separate methods:
- JsRuntime::poll_event_loop() - does single tick of event loop
- JsRuntime::run_event_loop() - runs event loop to completion
