This commit adds CJS/ESM interoperability when running in --compat mode.
Before executing files, they are analyzed and all CommonJS modules are
transformed on the fly to a ES modules. This is done by utilizing analyze_cjs()
functionality from deno_ast. After discovering exports and reexports, an ES
module is rendered and saved in memory for later use.
There's a caveat that all files ending with ".js" extension are considered as
CommonJS modules (unless there's a related "package.json" with "type": "module").
This commit adds "--trace-ops" flag to "deno test" subcommand.
This flag enables saving of stack traces for async ops, that before were always
saved. While the feature proved to be very useful it comes with a significant performance
hit, it's caused by excessive source mapping of stack frames.
This commit improves the error messages for the `deno test` async op
sanitizer. It does this in two ways:
- it uses handwritten error messages for each op that could be leaking
- it includes traces showing where each op was started
This "async op tracing" functionality is a new feature in deno_core.
It likely has a significant performance impact, which is why it is only
enabled in tests.
This commit adds "--output" to "deno coverage" subcommand.
It can be used instead of piping output to a file.
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit adds `readable` and `writable` properties to `Deno.File` and
`Deno.Conn`. This makes it very simple to use files and network sockets
with fetch or the native HTTP server.
Adds another callback to WebWorkerOptions that allows to execute
some modules before actual worker code executes. This allows to set up Node
global using std/node.
This commit makes the errors produced from the resource sanitizer much
more human readable. It does this by using real words rather than our
"resource names" when referring to resources, and by giving helpful
hints on how to clean up each of the resources.
This commit fixes an error when user deletes "window" global JS
variable. Instead of relying on "window" or "globalThis" to dispatch
"load" and "unload" events, we are default to global scope of the
worker.
V8 has supported `Intl.ListFormat` since version 7.2, but TypeScript doesn't
have typings for it yet. This PR manually adds those typings, copying them from
microsoft/TypeScript#47254.
Deno's module loader currently strips a shebang if a module file
starts with one. However, this is no longer necessary, since there is
a stage-3 TC39 that adds support for shebangs (or "hashbangs") to the
language (https://github.com/tc39/proposal-hashbang), and V8, `tsc`
and `swc` all support it.
Furthermore, stripping shebangs causes a correctness bug with JSON
modules, since a JSON file with a shebang should not parse as a JSON
module, yet it does with this stripping. This change fixes this.
This commit adds lint and fmt ignore directives to bundled
code as well as a comment stating that the code was bundled
and shouldn't be edited manually.
Covered ranges were not merged and thus it appeared that some lines
might be uncovered. To fix this I used "v8-coverage" that takes care
of merging the ranges properly. With this change, coverage collected
from a file by multiple entrypoints is now correctly calculated.
I ended up forking https://github.com/demurgos/v8-coverage and adding
"cli/tools/coverage/merge.rs" and "cli/tools/coverage/range_tree.rs".
This commit changes "deno coverage" command not to type check.
Instead of relying on infrastructure for module loading in "deno run";
the code now directly reaches into cache for original and transpiled
sources. In case sources are not available the error is returned to the
user, suggesting to first run "deno test --coverage" command.
This commit changes flow in inspector code to no longer require
"Runtime.runIfWaitingForDebugger" message to complete a handshake.
Even though clients like Chrome DevTools always send this message on startup,
it is against the protocol to require this message to start an inspector
session.
Instead "Runtime.runIfWaitingForDebugger" is required only when running with
"--inspect-brk" flag, which matches behavior of Node.js.
This commit fixes prompts printed to the terminal when
running with "--inspect" or "--inspect-brk" flags.
When debugger disconnects error is no longer printed as
users don't care about the reason debugger did disconnect.
A message suggesting to go to "chrome://inspect" is printed
if debugger is active.
Additionally and information that process is waiting for
debugger to connect is printed if running with "--inspect-brk"
flag.
This commit fixes inspector integration with "deno test" subcommand
by waiting for inspector sessions to connect if "--inspect-brk" flag
is passed.
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit adds proper support for import assertions and JSON modules.
Implementation of "core/modules.rs" was changed to account for multiple possible
module types, instead of always assuming that the code is an "ES module". In
effect "ModuleMap" now has knowledge about each modules' type (stored via
"ModuleType" enum). Module loading pipeline now stores information about
expected module type for each request and validates that expected type matches
discovered module type based on file's "MediaType".
Relevant tests were added to "core/modules.rs" and integration tests,
additionally multiple WPT tests were enabled.
There are still some rough edges in the implementation and not all WPT were
enabled, due to:
a) unclear BOM handling in source code by "FileFetcher"
b) design limitation of Deno's "FileFetcher" that doesn't download the same
module multiple times in a single run
Co-authored-by: Kitson Kelly <me@kitsonkelly.com>
std/http/server knows how to handle "Listener has been closed"
exceptions but not "operation canceled" errors.
Make "unix" listen sockets throw the same exception as "tcp" listen
sockets when the socket is closed and has a pending accept operation.
There is still a discrepancy when multiple accept requests are posted
but that's probably a less visible issue and something for another day.
Fixes #13033
This commit adds support for exporting RSA JWKs in the Web Crypto API.
It also does some minor fixes for RSA JWK imports.
Co-authored-by: Sean Michael Wykes <sean.wykes@nascent.com.br>
Fixes "op_set_exit_code" by sharing a single "Arc" between
all workers (via "op state") instead of having a "global" value stored in
"deno_runtime" crate. As a consequence setting an exit code is always
scoped to a tree of workers, instead of being overridable if there are
multiple worker tree (like in "deno test --jobs" subcommand).
Refactored "cli/main.rs" functions to return "Result<i32, AnyError>" instead
of "Result<(), AnyError>" so they can return exit code.
This change also makes the timers implementation closer to the spec, and
sets up the stage to implement AbortSignal.timeout() (whatwg/dom#1032).
Fixes #8965
Fixes #10974
Fixes #11398
Although not easy to replicate in the wild, the `deno test` op sanitizer
can fail when there are intervals that started before a test runs, since
the op sanitizer can end up running in the time between the timer op for
an interval's run resolves and the op for the next run starts.
This change fixes that by adding a new macrotask callback that will run
after the timer macrotask queue has drained. This ensures that there is
a timer op if there are any timers which are unresolved by the time the
op sanitizer runs.
Due to a bug in V8, terminating an isolate while a module with top-level
await is being evaluated would crash the process. This change makes it
so calling `worker.terminate()` will signal the worker to terminate at
the next iteration of the event loop, and it schedules a proper
termination of the worker's isolate after 2 seconds.
Although not easy to replicate in the wild, the `deno test` op sanitizer
can fail when there are intervals that started before a test runs, since
the op sanitizer can end up running in the time between the timer op for
an interval's run resolves and the op for the next run starts.
This change fixes that by adding a new macrotask callback that will run
after the timer macrotask queue has drained. This ensures that there is
a timer op if there are any timers which are unresolved by the time the
op sanitizer runs.
Set the exit code to use if none is provided to Deno.exit(), or when
Deno exits naturally.
Needed for process.exitCode Node compat. Paves the way for #12888.
Fetching of local files, added in #12545, returns a response with no
headers, including the `Content-Type` header. This currently makes it
not work with the WebAssembly streaming APIs, which require the response
to have a content type of `application/wasm`.
Since the only way to obtain a `Response` object with a non-empty `url`
field is via `fetch()`, this change changes the content type requirement
to only apply to responses whose url has the `file:` scheme.
Reduce the number of iterations from 1,024 to 128.
On my big bruiser of a desktop machine it already takes up close to a
minute to complete when nothing else is running so no way it's going to
finish in the allotted time on the CI.
The fact that the test used to pass may be indicative of a performance
regression somewhere but it's not clear to me when or where that would
have been introduced.
Fixes #12887.
This commit introduces "ProcState::maybe_resolver" field, which
stores a single instance of resolver for the whole lifetime of the
process, instead of creating these resolvers for each creation
of module graph. As a result, this resolver can be used in fallback
case where graph is not constructed (REPL, loading modules using
"require") unifying resolution logic.
This commit speeds up compat tests by using local copy of "deno_std"
instead of downloading it from https://deno.land for each test.
Additionally type checking is skipped.
The tests expect 3 publish notifications. It was possible for less than 3 to occur if two or more tasks set the diagnostics in the collection, exited the lock at the same time, then called `publish_diagnostics`
This allows resources to be "streams" by implementing read/write/shutdown. These streams are implicit since their nature (read/write/duplex) isn't known until called, but we could easily add another method to explicitly tag resources as streams.
`op_read/op_write/op_shutdown` are now builtin ops provided by `deno_core`
Note: this current implementation is simple & straightforward but it results in an additional alloc per read/write call
Closes #12556
Previously just a generic "error: No such file or directory (os error
2)" was printed. Now "`unzip` was not found on your PATH, please install
`unzip`" will be printed.
This adds `.code` attributes to errors returned by the op-layer, facilitating classifying OS errors and helping node-compat.
Similar to Node, these `.code` attributes are stringified names of unix ERRNOs, the mapping tables are generated by [tools/codegen_error_codes.js](https://gist.github.com/AaronO/dfa1106cc6c7e2a6ebe4dba9d5248858) and derived from libuv and rust's std internals
This commit changes formatting of elapsed time in test
runner output.
Instead of "XXXms", reporter outputs one of:
- "XXXms" for <1000ms
- "XXs" for <60s
- "XXXmYYs" for >=60s
This commit integrates import map and "classic" resolutions in
the "--compat" mode when using ES modules; in effect
"http:", "https:" and "blob:" imports now work in compat mode.
The algorithm works as follows:
1. If there's an import map, try to resolve using it and if succeeded
return the specifier
2. Try to resolve using "Node ESM resolution", and if succeeded return
the specifier
3. Fall back to regular ESM resolution
Closes #11826
**BREAKING CHANGE** this behaviour was disable when introduced in Deno 1.14/TypeScript 4.4. It will highlight code that unsafely handles variables that are caught, and will cause type errors in unsafe code.
Closes #11882
BREAKING CHANGE: Previously when `--location` was set, the unique storage key was derived from the the URL of the location instead of just the origin. This change correctly uses just the origin. This may cause previously persisted storage to change its key and data to not be available with the same location as before.
A `handshake()` method was added that returns when the TLS handshake is
complete. The `TlsListener` and `TlsConn` interfaces were added to
accomodate this new method.
Closes: #11759.
`fetch()` and client-side websocket used to support HTTP/2, but this
regressed in #11491. This patch reenables it by explicitly adding `h2`
and `http/1.1` to the list of ALPN protocols on the HTTP and websocket
clients.
A bug was fixed that could cause a hang when a method was
called on a TlsConn object that had thrown an exception earlier.
Additionally, a bug was fixed that caused TlsConn.write() to not
completely flush large buffers (>64kB) to the socket.
The public `TlsConn.handshake()` API is scheduled for inclusion in the
next minor release. See https://github.com/denoland/deno/pull/12467.
This commit adds CJS and ESM Node resolvers to the "--compat" mode.
The functionality is spread across "cli/compat" module and Node compatibility
layer in "deno_std/node"; this stems from the fact that ES module resolution
can only be implemented in Rust as it needs to directly integrated with
"deno_core"; however "deno_std/node" already provided CJS module resolution.
Currently this resolution is only active when running a files using
"deno run --compat --unstable <filename>", and is not available in other
subcommands, which will be changed in follow up commits.
Returns empty values in case of errors, source lines are non-essential anyway. These errors can happen e.g. when source files change at runtime. A warning is also printed to help us track when it happens in unexpected cases besides this.
Currently all async ops are polled lazily, which means that op
initialization code is postponed until control is yielded to the event
loop. This has some weird consequences, e.g.
```js
let listener = Deno.listen(...);
let conn_promise = listener.accept();
listener.close();
// `BadResource` is thrown. A reasonable error would be `Interrupted`.
let conn = await conn_promise;
```
JavaScript promises are expected to be eagerly evaluated. This patch
makes ops actually do that.
On OS X, the watcher sometimes witnesses the creation of it's own root
directory. Creating that directory using a sync op instead of an async
op sidesteps the issue.
This commit fixes problem with LSP where diagnostics coming
from "deno lint" don't respect configuration file.
LSP was changed to store "Option<ConfigFile>", "Option<LintConfig>"
and "Option<FmtConfig>" on "Inner"; as well as storing "Option<LintConfig>"
and "Option<FmtConfig>" on "StateSnapshot".
Co-authored-by: Kitson Kelly <me@kitsonkelly.com>
`Window`'s `self` property and `DedicatedWorkerGlobalScope`'s `name`
property are defined as Web IDL read-only attributes with the
`[Replaceable]` extended attribute, meaning that their setter will
redefine the property as a data property with the set value, rather than
changing some internal state. Deno currently defines them as read-only
data properties instead.
Given that Web IDL requires all attributes to be accessor properties
rather than data properties, but Deno exposes almost all of those
properties as either read-only or writable data properties, it makes
sense to expose `[Replaceable]` properties as writable as well – as is
already the case with `WindowOrWorkerGlobalScope`'s `performance`
property.
GET/HEAD requests can't have bodies according to `fetch` spec. This
commit changes the HTTP server to hide request bodies for requests with
GET or HEAD methods.
This commit annotates errors returned from FS Deno APIs to include
paths that were passed to the API calls.
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
WebAssembly modules compiled through `WebAssembly.compile()` and similar
non-streaming APIs don't have a URL associated to them, because they
have been compiled from a buffer source. In stack traces, V8 will use
a URL such as `wasm://wasm/d1c677ea`, with a hash of the module.
However, wasm modules compiled through streaming APIs, like
`WebAssembly.compileStreaming()`, do have a known URL, which can be
obtained from the `Response` object passed into the streaming APIs. And
as per the developer-facing display conventions in the WebAssembly
Web API spec, this URL should be used in stack traces. This change
implements that.
This commit adds automatic injection of Node globals when "--compat" flag
is present.
This is done by executing "https://deno.land/std/node/global.ts" as a "side module",
before main module is executed.
This commit makes "--compat" required to be used with "--unstable" flag, as some
of Node globals require unstable Deno APIs.
This commit adds "--compat" flag. When the flag is passed a set of mappings for
built-in Node modules is injected into the import map. If user doesn't
explicitly provide an import map (using "--import-map" flag) then a map is
created on the fly. If there are already existing mappings in import map that
would clash with built-in Node modules a set of diagnostics is printed to the
terminal with suggestions how to proceed.
This adds support for using in memory CA certificates for
`Deno.startTLS`, `Deno.connectTLS` and `Deno.createHttpClient`.
`certFile` is deprecated in `startTls` and `connectTls`, and removed
from `Deno.createHttpClient`.
Our oneshot receiver in `HyperService::call` would unwrap and panic, the `.await` on the oneshot receiver happens when the sender is dropped.
The sender is dropped in `op_http_response` because:
1. We take `ResponseSenderResource`
2. Then get `ConnResource` and early exit on failure (conn already closed)
3. The taken sender then gets dropped in this early exit before any response is sent over the channel
Fallbacking to returning a dummy response to hyper seems to be a fine quickfix
This panic could happen in the following cases:
- A non-fatal error being thrown from a worker, that doesn't terminate
the worker's execution, but propagates to the main thread without
being handled, and makes the main thread terminate.
- A nested worker being alive while its parent worker gets terminated.
- A race condition if the main event loop terminates the worker as part
of its last task, but the worker doesn't fully terminate before the
main event loop stops running.
This panic happens because a worker's event loop should have pending ops
as long as the worker isn't closed or terminated – but if an event loop
finishes running while it has living workers, its associated
`WorkerThread` structs will be dropped, closing the channels that keep
those ops pending.
This change adds a `Drop` implementation to `WorkerThread`, which
terminates the worker without waiting for a response. This fixes the
panic, and makes it so nested workers are automatically terminated once
any of their ancestors is closed or terminated.
This change also refactors a worker's termination code into a
`WorkerThread::terminate()` method.
Closes #11342.
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
