When a dynamically imported module gets resolved, any code that comes after an
await import() to that module will continue running. However, if that is the
last code in the evaluation of another dynamically imported module, that second
module will not resolve until the next iteration of the event loop, even though
it does not depend on the event loop at all.
When the event loop is being blocked by a long-running operation, such as a
long-running timer, or by an async op that might never end, such as with workers
or BroadcastChannels, that will result in the second dynamically imported module
not being resolved for a while, or ever.
This change fixes this by running the dynamic module loading steps in a loop
until no more dynamic modules can be resolved.
This commit adds proper support for import assertions and JSON modules.
Implementation of "core/modules.rs" was changed to account for multiple possible
module types, instead of always assuming that the code is an "ES module". In
effect "ModuleMap" now has knowledge about each modules' type (stored via
"ModuleType" enum). Module loading pipeline now stores information about
expected module type for each request and validates that expected type matches
discovered module type based on file's "MediaType".
Relevant tests were added to "core/modules.rs" and integration tests,
additionally multiple WPT tests were enabled.
There are still some rough edges in the implementation and not all WPT were
enabled, due to:
a) unclear BOM handling in source code by "FileFetcher"
b) design limitation of Deno's "FileFetcher" that doesn't download the same
module multiple times in a single run
Co-authored-by: Kitson Kelly <me@kitsonkelly.com>
Currently all async ops are polled lazily, which means that op
initialization code is postponed until control is yielded to the event
loop. This has some weird consequences, e.g.
```js
let listener = Deno.listen(...);
let conn_promise = listener.accept();
listener.close();
// `BadResource` is thrown. A reasonable error would be `Interrupted`.
let conn = await conn_promise;
```
JavaScript promises are expected to be eagerly evaluated. This patch
makes ops actually do that.
This commit fixes a problem where loading and executing multiple
modules leads to all of the having "import.meta.main" set to true.
Following Rust APIs were deprecated:
- deno_core::JsRuntime::load_module
- deno_runtime::Worker::execute_module
- deno_runtime::WebWorker::execute_module
Following Rust APIs were added:
- deno_core::JsRuntime::load_main_module
- deno_core::JsRuntime::load_side_module
- deno_runtime::Worker::execute_main_module
- deno_runtime::Worker::execute_side_module
- deno_runtime::WebWorker::execute_main_module
Trying to load multiple "main" modules into the runtime now results in an
error. If user needs to load additional "non-main" modules they should use
APIs for "side" module.
Oneshot is more appropriate because mod_evaluate() only sends a single
value.
It also makes it easier to use it correctly. As an embedder, I wasn't
sure if I'm expected to drain the channel or not.
This commit changes implementation of module loading in "deno_core"
to track all currently fetched modules across all existing module loads.
In effect a bug that caused concurrent dynamic imports referencing the
same module to fail is fixed.
This commit renames "JsRuntime::execute" to "JsRuntime::execute_script". Additionally
same renames were applied to methods on "deno_runtime::Worker" and
"deno_runtime::WebWorker".
A new macro was added to "deno_core" called "located_script_name" which
returns the name of Rust file alongside line no and col no of that call site.
This macro is useful in combination with "JsRuntime::execute_script"
and allows to provide accurate place where "one-off" JavaScript scripts
are executed for internal runtime functions.
Co-authored-by: Nayeem Rahman <nayeemrmn99@gmail.com>
This commit changes module loading implementation in "deno_core"
to call "ModuleLoader::prepare" hook only once per entry point.
This is done to avoid multiple type checking of the same code
in case of duplicated dynamic imports.
Relevant code in "cli/module_graph.rs" was updated as well.
This commit moves implementation of "JsRuntimeInspector" to "deno_core" crate.
To achieve that following changes were made:
* "Worker" and "WebWorker" no longer own instance of "JsRuntimeInspector",
instead it is now owned by "deno_core::JsRuntime".
* Consequently polling of inspector is no longer done in "Worker"/"WebWorker",
instead it's done in "deno_core::JsRuntime::poll_event_loop".
* "deno_core::JsRuntime::poll_event_loop" and "deno_core::JsRuntime::run_event_loop",
now accept "wait_for_inspector" boolean that tells if event loop should still be
"pending" if there are active inspector sessions - this change fixes the problem
that inspector disconnects from the frontend and process exits once the code has
stopped executing.
This commit moves bulk of the logic related to module loading
from "JsRuntime" to "ModuleMap".
Next steps are to rewrite the actual loading logic (represented by
"RecursiveModuleLoad") to be a part of "ModuleMap" as well --
that way we will be able to track multiple module loads from within
the map which should help me solve the problem of concurrent
loads (since all info about currently loading/loaded modules will
be contained in the ModuleMap, so we'll be able to know if actually
all required modules have been loaded).
General cleanup of module loading code, tried to reduce indentation in various methods
on "JsRuntime" to improve readability.
Added "JsRuntime::handle_scope" helper function, which returns a "v8::HandleScope".
This was done to reduce a code pattern that happens all over the "deno_core".
Additionally if event loop hangs during loading of dynamic modules a list of
currently pending dynamic imports is printed.
This commit rewrites implementation of "JsRuntime::mod_evaluate".
Event loop is no longer polled automatically and users must manually
drive event loop forward after calling "mod_evaluate".
Co-authored-by: Nayeem Rahman <nayeemrmn99@gmail.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Implementors of `deno_core::JsRuntime` might want to do additional actions
during each turn of event loop, eg. `deno_runtime::Worker` polls inspector,
`deno_runtime::WebWorker` receives/dispatches messages from/to worker host.
Previously `JsRuntime::mod_evaluate` was implemented in such fashion that it
only polled `JsRuntime`'s event loop. This behavior turned out to be wrong
in the example of `WebWorker` which couldn't receive/dispatch messages because
its implementation of event loop was never called.
This commit rewrites "mod_evaluate" to return a handle to receiver that resolves
when module's promise resolves. It is now implementors responsibility to poll
event loop after calling `mod_evaluate`.
This commit does major refactor of "Worker" and "WebWorker",
in order to decouple them from "ProgramState" and "Flags".
The main points of interest are "create_main_worker()" and
"create_web_worker_callback()" functions which are responsible
for creating "Worker" and "WebWorker" in CLI context.
As a result it is now possible to factor out common "runtime"
functionality into a separate crate.
This commit adds `FsModuleLoader` to `deno_core`, which implements
`ModuleLoader` trait. It is used when creating a runtime that supports
module loading from filesystem.
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit refactors "deno_core::Modules" structure to not depend on
"get_identity_hash" function to identify modules, but instead use default
hash implementation.
This commit changes implementation of top-level-await in "deno_core".
Previously promise returned from module evaluation was not awaited,
leading to out-of-order execution of modules that have TLA. It's been
fixed by changing "JsRuntime::mod_evaluate" to be an async function
that resolves when the promise returned from module evaluation also
resolves. When waiting for promise resolution event loop is polled
repeatedly, until there are no more dynamic imports or pending
ops.
* Revert "refactor: Worker is not a Future (#7895)"
This reverts commit f4357f0ff9.
* Revert "refactor(core): JsRuntime is not a Future (#7855)"
This reverts commit d8879feb8c.
* Revert "fix(core): module execution with top level await (#7672)"
This reverts commit c7c7677825.
This commit fixes implementation of top level await in "deno_core".
Previously promise returned from module execution was ignored causing to execute
modules out-of-order.
With this commit promise returned from module execution is stored on "JsRuntime"
and event loop is polled until the promise resolves.
This makes use of a default referrer when its empty in repl mode so that
dynamic imports work in the global evaluation context.
Co-authored-by: Bartek Iwanczuk <biwanczuk@gmail.com>
This commit adds support for stack traces in "deno_core".
Implementation of "Display" trait for "JsError" has been updated
and in consequence "deno_core::js_check" became obsolete and
removed.
- remove "CliState.workers" and "CliState.next_worker_id", instead
store them on "OpState" using type aliases.
- remove "CliState.global_timer" and "CliState.start_time", instead
store them on "OpState" using type aliases.
- remove "CliState.is_internal", instead pass it to Worker::new
- move "CliState::permissions" to "OpState"
- move "CliState::main_module" to "OpState"
- move "CliState::global_state" to "OpState"
- move "CliState::check_unstable()" to "GlobalState"
- change "cli_state()" to "global_state()"
- change "deno_core::ModuleLoader" trait to pass "OpState" to callbacks
- rename "CliState" to "CliModuleLoader"
Removes:
- "deno_core::StartupData"
- "deno_core::Script"
- "deno_core::OwnedScript"
Changes to "JsRuntime":
- remove "new_with_loader()"
- remove "with_heap_limits()"
- rename "IsolateOptions" to "RuntimeOptions" and make public
- "JsRuntime::new()" takes "RuntimeOptions" as a single param
