When a dynamically imported module gets resolved, any code that comes after an
await import() to that module will continue running. However, if that is the
last code in the evaluation of another dynamically imported module, that second
module will not resolve until the next iteration of the event loop, even though
it does not depend on the event loop at all.
When the event loop is being blocked by a long-running operation, such as a
long-running timer, or by an async op that might never end, such as with workers
or BroadcastChannels, that will result in the second dynamically imported module
not being resolved for a while, or ever.
This change fixes this by running the dynamic module loading steps in a loop
until no more dynamic modules can be resolved.
This commit adds support for unstable FFI
callbacks. A callback is registered using
the `Deno.UnsafeCallback` API.
The backing memory for the callback can
be disposed of using `Deno.UnsafeCallback#close`.
It is not safe to pass the callback after calling
close.
Callbacks from other than the isolate thread
are not supported.
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Co-authored-by: Bert Belder <bertbelder@gmail.com>
This commit updates the custom inspect function for URL objects
to pass the inspect options through so that the context is
propagated and the resulting indentation is correct.
Fixes: https://github.com/denoland/deno/issues/14171
This commit changes default default behavior of type checking
for several subcommands.
Instead of type checking and reporting type errors only for local
files, the type checking is skipped entirely. Type checking can
still be enabled using the "--check" flag.
Following subcomands are affected:
- deno cache
- deno install
- deno eval
- deno run
This commit adds "Deno.core.setFormatExceptionCallback" which
can be used to provide custom formatting for errors. It is useful
in cases when user throws something that is non-Error (eg.
a string, plain object, etc).
This commit changes default mode of type-checking to "local"
and adds "--check" flag to following subcommands:
- deno bench
- deno bundle
- deno cache
- deno compile
- deno eval
- deno install
- deno test
This commit removes "WorkerOptions.deno" option as a boolean,
as well as "WorkerOptions.deno.namespace" settings. Starting
with this commit all workers have access to "Deno" namespace
by default.
This commit fixes source maps for files that contain emojis.
This is done by updating "deno_ast" to "0.14.1" for the case
of "--no-check" flag (ie using SWC emit) and by overriding
TSC's default base64 encoder (which turned out to be buggy)
for the type checking case.
This flag disables loading of configuration file, ie. it will not be
automatically discovered and loaded. Of course this flag conflicts
with "--config" flag and they cannot be used together.
This commit adds better reporting of uncaught errors
in top level scope of testing files. This change affects
both console runner as well as LSP runner.
Calling `worker.terminate()` used to kill the worker's isolate and
then block until the worker's thread finished. This blocks the calling
thread if the worker's event loop was blocked in a sync op (as with
`Deno.sleepSync`), which wasn't realized at the time, but since the
worker's isolate was killed at that moment, it would not block the
calling thread if the worker was in a JS endless loop.
However, in #12831, in order to work around a V8 bug, worker
termination was changed to first set a signal to let the worker event
loop know that termination has been requested, and only kill the
isolate if the event loop has not finished after 2 seconds. However,
this change kept the blocking, which meant that JS endless loops in
the worker now blocked the parent for 2 seconds.
As it turns out, after #12831 it is fine to signal termination and
even kill the worker's isolate without waiting for the thread to
finish, so this change does that. However, that might leave the async
ops that receive messages and control data from the worker pending
after `worker.terminate()`, which leads to odd results from the op
sanitizer. Therefore, we set up a `CancelHandler` to cancel those ops
when the worker is terminated.
This commit:
- removes "fmt_errors::PrettyJsError" in favor of "format_js_error" fn
- removes "deno_core::JsError::create" and
"deno_core::RuntimeOptions::js_error_create_fn"
- adds new option to "deno_runtime::ops::worker_host::init"
This commit changes "deno bench" subcommand, by updating
the "Deno.bench" API as follows:
- remove "Deno.BenchDefinition.n"
- remove "Deno.BenchDefintion.warmup"
- add "Deno.BenchDefinition.group"
- add "Deno.BenchDefintion.baseline"
This is done because bench cases are no longer run fixed amount
of iterations, but instead they are run until there is difference between
subsequent runs that is statistically insiginificant.
Additionally, console reporter was rewritten completely, to looks
similar to "hyperfine" reporter.
This commit changes "deno test" to filter out stack frames if it is beneficial to the user.
This is the case when there are stack frames coming from "internal" code
below frames coming from user code.
Co-authored-by: Nayeem Rahman <nayeemrmn99@gmail.com>
This commit fixes and edge case, where testing/benching code could pledge new
permission set before restoring the previous pledge.
Appropriate panics were added and tests that assert that process is killed
in case of "recursive pledge".
This commit rewrites test runner to send structured error data from JavaScript
to Rust instead of passing strings. This will allow to customize display of errors
in test report (which will be addressed in follow up commits).
This commit adds "aggregated" field to "deno_core::JsError" that stores
instances of "JsError" recursively to properly handle "AggregateError"
formatting. Appropriate logics was added to "PrettyJsError" and
"console" API to format AggregateErrors.
Co-authored-by: Nayeem Rahman <nayeemrmn99@gmail.com>
The following transformations gradually faced by "JsError" have all been
moved up front to "JsError::from_v8_exception()":
- finding the first non-"deno:" source line;
- moving "JsError::script_resource_name" etc. into the first error stack
in case of syntax errors;
- source mapping "JsError::script_resource_name" etc. when wrapping
the error even though the frame locations are source mapped earlier;
- removing "JsError::{script_resource_name,line_number,start_column,end_column}"
entirely in favour of "js_error.frames.get(0)".
We also no longer pass a js-side callback to "core/02_error.js" from cli.
I avoided doing this on previous occasions because the source map lookups
were in an awkward place.
This commit changes "deno test" to better denote user output coming
from test cases.
This is done by printing "---- output ----" and "---- output end ----"
markers if an output is produced. The output from "console" and
"Deno.core.print" is captured, as well as direct writes to "Deno.stdout"
and "Deno.stderr".
To achieve that new APIs were added to "deno_core" crate, that allow
to replace an existing resource with a different one (while keeping resource
ids intact). Resources for stdout and stderr are replaced by pipes.
Co-authored-by: David Sherret <dsherret@gmail.com>
Following changes were done in this commit:
- remove "test" prefix before each test
- use gray color for "running N tests from ..." prompt
- use relative path or remote URL instead of full URL in "running N tests from ..." prompt
- in "failures" section, add file path/remote URL before the test name
Co-authored-by: Yoshiya Hinosawa <stibium121@gmail.com>
This commit adds new "deno check" subcommand.
Currently it is an alias for "deno cache" with the difference that remote
modules don't emit TS diagnostics by default.
Prints warning for "deno run" subcommand if "--check" flag is not present
and there's no "--no-check" flag. Adds "DENO_FUTURE_CHECK" env
variable that allows to opt into new behavior now.
`handleWasmStreaming` is the function that provides the binding with
the `fetch` API needed for `WebAssembly.instantiateStreaming()` and
`WebAssembly.compileStreaming()`. When I implemented it in #11200, I
thought V8 was calling these functions with the argument of the
`WebAssembly` streaming functions, without doing any resolving, and so
`handleWasmStreaming` awaits for the parameter to resolve. However,
as discovered in
https://github.com/denoland/deno/issues/13917#issuecomment-1065805565,
V8 does in fact resolve the parameter if it's a promise (and handles
rejections arising from that).
This change removes the `async` IIFE inside `handleWasmStreaming`,
letting initial errors be handled synchronously (which will however
not throw synchronously from the `WebAssembly` namespace functions).
Awaiting is still necessary for reading the bytes of the response,
though, and so there is an `async` IIFE for that.
When an exception is thrown during the processing of streaming WebAssembly,
`op_wasm_streaming_abort` is called. This op calls into V8, which synchronously
rejects the promise and calls into the promise rejection handler, if applicable.
But calling an op borrows the isolate's `JsRuntimeState` for the duration of the
op, which means it is borrowed when V8 calls into `promise_reject_callback`,
which tries to borrow it again, panicking.
This change changes `op_wasm_streaming_abort` from an op to a binding
(`Deno.core.abortWasmStreaming`). Although that binding must borrow the
`JsRuntimeState` in order to access the `WasmStreamingResource` stored in the
`OpTable`, it also takes ownership of that `WasmStreamingResource` instance,
which means it can drop any borrows of the `JsRuntimeState` before calling into
V8.
This commit adds "deno bench" subcommand and "Deno.bench()"
API that allows to register bench cases.
The API is modelled after "Deno.test()" and "deno test" subcommand.
Currently the output is rudimentary and bench cases and not
subject to "ops" and "resource" sanitizers.
Co-authored-by: evan <github@evan.lol>
This commit fixes CJS/ESM interop in compat mode for dynamically
imported modules.
"ProcState::prepare_module_load" was changed to accept a list
of "graph roots" without associated "module kind". That module kind
was always hardcoded to "ESM" which is not true for CJS/ESM interop -
a CommonJs module might be imported using "import()" function. In
such case the root of the graph should have "CommonJs" module kind
instead of "ESM".
This commit adds CJS/ESM interoperability when running in --compat mode.
Before executing files, they are analyzed and all CommonJS modules are
transformed on the fly to a ES modules. This is done by utilizing analyze_cjs()
functionality from deno_ast. After discovering exports and reexports, an ES
module is rendered and saved in memory for later use.
There's a caveat that all files ending with ".js" extension are considered as
CommonJS modules (unless there's a related "package.json" with "type": "module").
This commit adds "--trace-ops" flag to "deno test" subcommand.
This flag enables saving of stack traces for async ops, that before were always
saved. While the feature proved to be very useful it comes with a significant performance
hit, it's caused by excessive source mapping of stack frames.
This commit improves the error messages for the `deno test` async op
sanitizer. It does this in two ways:
- it uses handwritten error messages for each op that could be leaking
- it includes traces showing where each op was started
This "async op tracing" functionality is a new feature in deno_core.
It likely has a significant performance impact, which is why it is only
enabled in tests.
Adds another callback to WebWorkerOptions that allows to execute
some modules before actual worker code executes. This allows to set up Node
global using std/node.
This commit makes the errors produced from the resource sanitizer much
more human readable. It does this by using real words rather than our
"resource names" when referring to resources, and by giving helpful
hints on how to clean up each of the resources.
This commit fixes an error when user deletes "window" global JS
variable. Instead of relying on "window" or "globalThis" to dispatch
"load" and "unload" events, we are default to global scope of the
worker.
Deno's module loader currently strips a shebang if a module file
starts with one. However, this is no longer necessary, since there is
a stage-3 TC39 that adds support for shebangs (or "hashbangs") to the
language (https://github.com/tc39/proposal-hashbang), and V8, `tsc`
and `swc` all support it.
Furthermore, stripping shebangs causes a correctness bug with JSON
modules, since a JSON file with a shebang should not parse as a JSON
module, yet it does with this stripping. This change fixes this.
This commit adds lint and fmt ignore directives to bundled
code as well as a comment stating that the code was bundled
and shouldn't be edited manually.
Covered ranges were not merged and thus it appeared that some lines
might be uncovered. To fix this I used "v8-coverage" that takes care
of merging the ranges properly. With this change, coverage collected
from a file by multiple entrypoints is now correctly calculated.
I ended up forking https://github.com/demurgos/v8-coverage and adding
"cli/tools/coverage/merge.rs" and "cli/tools/coverage/range_tree.rs".
This commit fixes inspector integration with "deno test" subcommand
by waiting for inspector sessions to connect if "--inspect-brk" flag
is passed.
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit adds proper support for import assertions and JSON modules.
Implementation of "core/modules.rs" was changed to account for multiple possible
module types, instead of always assuming that the code is an "ES module". In
effect "ModuleMap" now has knowledge about each modules' type (stored via
"ModuleType" enum). Module loading pipeline now stores information about
expected module type for each request and validates that expected type matches
discovered module type based on file's "MediaType".
Relevant tests were added to "core/modules.rs" and integration tests,
additionally multiple WPT tests were enabled.
There are still some rough edges in the implementation and not all WPT were
enabled, due to:
a) unclear BOM handling in source code by "FileFetcher"
b) design limitation of Deno's "FileFetcher" that doesn't download the same
module multiple times in a single run
Co-authored-by: Kitson Kelly <me@kitsonkelly.com>
Fixes "op_set_exit_code" by sharing a single "Arc" between
all workers (via "op state") instead of having a "global" value stored in
"deno_runtime" crate. As a consequence setting an exit code is always
scoped to a tree of workers, instead of being overridable if there are
multiple worker tree (like in "deno test --jobs" subcommand).
Refactored "cli/main.rs" functions to return "Result<i32, AnyError>" instead
of "Result<(), AnyError>" so they can return exit code.
This change also makes the timers implementation closer to the spec, and
sets up the stage to implement AbortSignal.timeout() (whatwg/dom#1032).
Fixes #8965
Fixes #10974
Fixes #11398
Although not easy to replicate in the wild, the `deno test` op sanitizer
can fail when there are intervals that started before a test runs, since
the op sanitizer can end up running in the time between the timer op for
an interval's run resolves and the op for the next run starts.
This change fixes that by adding a new macrotask callback that will run
after the timer macrotask queue has drained. This ensures that there is
a timer op if there are any timers which are unresolved by the time the
op sanitizer runs.
Due to a bug in V8, terminating an isolate while a module with top-level
await is being evaluated would crash the process. This change makes it
so calling `worker.terminate()` will signal the worker to terminate at
the next iteration of the event loop, and it schedules a proper
termination of the worker's isolate after 2 seconds.
Although not easy to replicate in the wild, the `deno test` op sanitizer
can fail when there are intervals that started before a test runs, since
the op sanitizer can end up running in the time between the timer op for
an interval's run resolves and the op for the next run starts.
This change fixes that by adding a new macrotask callback that will run
after the timer macrotask queue has drained. This ensures that there is
a timer op if there are any timers which are unresolved by the time the
op sanitizer runs.
Set the exit code to use if none is provided to Deno.exit(), or when
Deno exits naturally.
Needed for process.exitCode Node compat. Paves the way for #12888.
Reduce the number of iterations from 1,024 to 128.
On my big bruiser of a desktop machine it already takes up close to a
minute to complete when nothing else is running so no way it's going to
finish in the allotted time on the CI.
The fact that the test used to pass may be indicative of a performance
regression somewhere but it's not clear to me when or where that would
have been introduced.
Fixes #12887.
This commit introduces "ProcState::maybe_resolver" field, which
stores a single instance of resolver for the whole lifetime of the
process, instead of creating these resolvers for each creation
of module graph. As a result, this resolver can be used in fallback
case where graph is not constructed (REPL, loading modules using
"require") unifying resolution logic.
This commit changes formatting of elapsed time in test
runner output.
Instead of "XXXms", reporter outputs one of:
- "XXXms" for <1000ms
- "XXs" for <60s
- "XXXmYYs" for >=60s
This commit integrates import map and "classic" resolutions in
the "--compat" mode when using ES modules; in effect
"http:", "https:" and "blob:" imports now work in compat mode.
The algorithm works as follows:
1. If there's an import map, try to resolve using it and if succeeded
return the specifier
2. Try to resolve using "Node ESM resolution", and if succeeded return
the specifier
3. Fall back to regular ESM resolution
Closes #11826
**BREAKING CHANGE** this behaviour was disable when introduced in Deno 1.14/TypeScript 4.4. It will highlight code that unsafely handles variables that are caught, and will cause type errors in unsafe code.
