Part of #25028.
Our underlying read/write operations in `io` assume the buffer is a
Uint8Array, but we were passing in other typed arrays (in the case above
it was `Int8Array`).
There is no constructor code when creating an inspector `Session`
instance in Node. Also get rid of some symbols which should've been
private properties. This PR doesn't yet add any new implementations
though as these are mostly cosmetic changes.
For some reason we didn't register the `node:inspector` module, which
lead to a panic when trying to import it. This PR registers it.
Related: https://github.com/denoland/deno/issues/25004
This is commonly used to register loading non standard file types. But
some libs also register TS loaders which Deno supports natively, like
the npm `payload` package. This PR unblocks those.
Fixes https://github.com/denoland/deno/issues/24902
- Return auth tag for GCM ciphers from auto padding shortcircuit
- Use _ring_ for ed25519 signing
---------
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
This PR ensures that we forward a `rename` event in our file watcher.
The rust lib we use combines that with the `modify` event.
This fixes a compatibility issue with Node too, which sends the `rename`
event as well.
Fixes https://github.com/denoland/deno/issues/24880
This completely rewrites how we handle key material in ext/node. Changes
in this
PR:
- **Signing**
- RSA
- RSA-PSS 🆕
- DSA 🆕
- EC
- ED25519 🆕
- **Verifying**
- RSA
- RSA-PSS 🆕
- DSA 🆕
- EC 🆕
- ED25519 🆕
- **Private key import**
- Passphrase encrypted private keys 🆕
- RSA
- PEM
- DER (PKCS#1) 🆕
- DER (PKCS#8) 🆕
- RSA-PSS
- PEM
- DER (PKCS#1) 🆕
- DER (PKCS#8) 🆕
- DSA 🆕
- EC
- PEM
- DER (SEC1) 🆕
- DER (PKCS#8) 🆕
- X25519 🆕
- ED25519 🆕
- DH
- **Public key import**
- RSA
- PEM
- DER (PKCS#1) 🆕
- DER (PKCS#8) 🆕
- RSA-PSS 🆕
- DSA 🆕
- EC 🆕
- X25519 🆕
- ED25519 🆕
- DH 🆕
- **Private key export**
- RSA 🆕
- DSA 🆕
- EC 🆕
- X25519 🆕
- ED25519 🆕
- DH 🆕
- **Public key export**
- RSA
- DSA 🆕
- EC 🆕
- X25519 🆕
- ED25519 🆕
- DH 🆕
- **Key pair generation**
- Overhauled, but supported APIs unchanged
This PR adds a lot of new individual functionality. But most importantly
because
of the new key material representation, it is now trivial to add new
algorithms
(as shown by this PR).
Now, when adding a new algorithm, it is also widely supported - for
example
previously we supported ED25519 key pair generation, but we could not
import,
export, sign or verify with ED25519. We can now do all of those things.
- upgrade to v8 12.8
- optimizes DataView bigint methods
- fixes global interceptors
- includes CPED methods for ALS
- fix global resolution
- makes global resolution consistent using host_defined_options.
originally a separate patch but due to the global interceptor bug it
needs to be included in this pr for all tests to pass.
Fixes https://github.com/denoland/deno/issues/24756. Fixes
https://github.com/denoland/deno/issues/24796.
This also gets vitest working when using
[`--pool=forks`](https://vitest.dev/guide/improving-performance#pool)
(which is the default as of vitest 2.0). Ref
https://github.com/denoland/deno/issues/23882.
---
This PR resolves a handful of issues with child_process IPC. In
particular:
- We didn't support sending typed array views over IPC
- Opening an IPC channel resulted in the event loop never exiting
- Sending a `null` over IPC would terminate the channel
- There was some UB in the read implementation (transmuting an `&[u8]`
to `&mut [u8]`)
- The `send` method wasn't returning anything, so there was no way to
signal backpressure (this also resulted in the benchmark
`child_process_ipc.mjs` being misleading, as it tried to respect
backpressure. That gave node much worse results at larger message sizes,
and gave us much worse results at smaller message sizes).
- We weren't setting up the `channel` property on the `process` global
(or on the `ChildProcess` object), and also didn't have a way to
ref/unref the channel
- Calling `kill` multiple times (or disconnecting the channel, then
calling kill) would throw an error
- Node couldn't spawn a deno subprocess and communicate with it over IPC
This commit duplicates ops from "ext/fetch" to "ext/node" to
kick off a bigger rewrite of "node:http".
Most of duplication is temporary and will be removed as these
ops evolve.
Support `MessagePort.once` in Node mode and enable relevant
`worker_threads` test. Noticed that another Node test was passing as
well, so I enabled that too.
The intent is that those tests will be executed, but our check that the
files are up to date won't overwrite the contents of the tests. This is
useful when a test needs some manual edits to work.
It turns out we weren't actually running them.
---
This ended up turning into a couple of small bug fixes to get the tests
passing:
- We weren't canonicalizing the exec path properly (it sometimes still
had `..` or `.` in it)
- We weren't accepting strings in `process.exit`
There was one failure I couldn't figure out quickly, so I disabled the
test for now, and filed a follow up issue: #24694
We were missing an import of `emitWarning` in our streams
implementation. The code prior to this PR assumed that `process` would
be available as a global.
Fixes https://github.com/denoland/deno/issues/23709
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Use `access` on *nix and `GetFileAttributesW` on Windows.
[Benchmark](https://paste.divy.work/p/-gq8Ark.js):
```
$ deno run -A bench.mjs # main (568dd)
existsSync: 8980.636629ms
$ target/release/deno run -A bench.mjs # this PR
existsSync: 6448.7604519999995ms
$ bun bench.mjs
existsSync: 6562.88671ms
$ node bench.mjs
existsSync: 7740.064653ms
```
Ref https://github.com/denoland/deno/pull/24434#discussion_r1679777912
This PR stubs `perf_hooks.eventLoopUtilization` to make the tests of
[hapi](https://github.com/hapijs/hapi) start. Previously, they'd all
error because of this function throwing a not implemented error. This
brings down the test failures in their suite from 982 to 68 failures.
Adds support for running npm package lifecycle scripts, opted into via a
new `--allow-scripts` flag.
With this PR, when running `deno cache` (or `DENO_FUTURE=1 deno
install`) you can specify the `--allow-scripts=pkg1,pkg2` flag to run
lifecycle scripts attached to the given packages.
Note at the moment this only works when `nodeModulesDir` is true (using
the local resolver).
When a package with un-run lifecycle scripts is encountered, we emit a
warning suggesting things may not work and to try running lifecycle
scripts. Additionally, if a package script implicitly requires
`node-gyp` and it's not found on the system, we emit a warning.
Extra things in this PR:
- Extracted out bits of `task.rs` into a separate module for reuse
- Added a couple fields to `process.config` in order to support
`node-gyp` (it relies on a few variables being there)
- Drive by fix to downloading new npm packages to test registry
---
TODO:
- [x] validation for allow-scripts args (make sure it looks like an npm
package)
- [x] make allow-scripts matching smarter
- [ ] figure out what issues this closes
---
Review notes:
- This adds a bunch of deps to our test registry due to using
`node-gyp`, so it's pretty noisy
The implementation for `assert.throws()` from `node:assert` didn't work
when the expected value was an `Error` constructor. In this case the
thrown error should checked if it's an instance of said constructor.
Fixes https://github.com/denoland/deno/issues/24464
Previously we had many different code paths all
handling digests in different places, all with
wildly different digest support. This commit
rewrites this to use a single digest handling
mechanism for all digest operations.
It adds various aliases for digest algorithms,
like node does. For example
`sha1WithRSAEncryption` is an alias for `sha1`.
It also adds support for `md5-sha1` digests in
various places.
It's perfectly valid to access `server.address()` before calling
`.listen()`. Until a server actively listens on a socket Node will
return `null` here, but we threw a "Cannot access property 'port' of
undefined" instead.
This was discovered when inspecting failures in Koa's test suite with
Deno.
Part of #18218
- Adds `fs.lutimes` and `fs.lutimesSync` to our node polyfills. To do
this I added methods to the `FileSystem` trait + ops to expose the
functionality to JS.
- Exports `fs._toUnixTimestamp`. Node exposes an internal util
`toUnixTimestamp` from the fs module to be used by unit tests (so we
need it for the unit test to pass unmodified). It's weird because it's
only supposed to be used internally but it's still publicly accessible
- Matches up error handling and timestamp handling for fs.futimes and
fs.utimes with node
- Enables the node_compat utimes test - this exercises futimes, lutimes,
and utimes.
Changes in this PR:
- Added new fixed size hash algorithms (blake2b512, blake2s256,
sha512-224, sha512-256, sha3-224, sha3-256, sha3-384, sha3-512, sm3)
- Added variable size hash algorithms (the concept), with the algorithms
shake128 and shake256
- Use cppgc instead of resources for the hasher
- Enable Node's crypto.Hash tests and fix found bugs
- add fallback impls of external string apis which always copy. after
upstream changes to rusty_v8 we can support non-copying api as well.
- `napi_get_buffer_data` needs to work on all TypedArray instances.
- Fixes: https://github.com/denoland/deno/issues/24209
- `target_defaults.default_configuration` is used by some modules to
find the corresponding node file from node-gyp
- `node_api_get_module_filename` expects the filename to be a `file:`
url.
Previously res.setHeader("foo", ["bar", "baz"]) added a single header
with a value of `bar,baz`. Really this should add two separate headers.
This is visible in `set-cookie` for example.
This commits adds the ability to set a would-be exit code
for the Deno process without forcing an immediate exit,
through the new `Deno.exitCode` API.
- **Implements `Deno.exitCode` getter and setter**: Adds support for
setting
and retrieving a would-be exit code via `Deno.exitCode`.
This allows for asynchronous cleanup before process termination
without immediately exiting.
- **Ensures type safety**: The setter for `Deno.exitCode` validates that
the provided value is a number, throwing a TypeError if not, to ensure
that
only valid exit codes are set.
Closes to #23605
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit changes `gzip` compression in `Deno.serve` API to flush data
after each write. There's a slight performance regression, but provided
test shows a scenario that was not possible before.
---------
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
We didn't honour the `position` options of `fd.read` and `fd.write`
because we checked if the buffer is of type `Buffer` instead of just
`Uint8Array`. Node does the latter. In doing so I noticed that the file
handle id was written to a public property which it definitely shouldn't
be. This was probably a typo.
Fixes https://github.com/denoland/deno/issues/23707
We didn't support the `throwIfNoEntry` option for Node's `fs.lstatSync`
method. Note that the async variant doesn't have this option.
Fixes https://github.com/denoland/deno/issues/23996
Node sets the default HTTP response status code to 200 on the
`ServerResponse`. We initialised it as `undefined` before which caused a
problem with 11ty's dev server.
Thanks to @vrugtehagel for reporting this issue and finding the correct
fix as well 🎉
Fixes https://github.com/denoland/deno/issues/23970
The flag lets us exit from read loop without throwing an error when
the stream is cancelled.
This fixes gRPC cancellation example.
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This stubs `findSourceMap` in `node:module` by always returning
`undefined` as if it never found a source map. This unblocks the `ava`
test runner.
Fixes https://github.com/denoland/deno/issues/18666
Some npm libraries like `signal-exit` rely on the length of the listener
array returned by `process.listeners("SIGNT")` to be correct to
function. We weren't tracking `SIG*` events there, which broke those npm
libraries.
Fixes https://github.com/denoland/deno/issues/22892
the last commit had a regression, where it removed this branch, I
haven't tested the code but I think it should work
---------
Signed-off-by: Bedis Nbiba <bedisnbiba@gmail.com>
Popular test runners like Jest instantiate a new `Process` object
themselves and expect the class constructor to be callable without the
`new` keyword. This PR refactors our `Process` class implementation from
a proper ES2015 class to an ES5-style class which can be invoked both
with and without the `new` keyword like in Node.
Fixes https://github.com/denoland/deno/issues/23863
gRPC depends only on the END_STREAM flag to emit "trailers" event which
is responsible to propagate the errors correctly. This patch uses
Body::is_end_stream() to determine if a stream will end and set the
END_STREAM flag.
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
<!--
Before submitting a PR, please read
https://docs.deno.com/runtime/manual/references/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Fixes https://github.com/denoland/deno/issues/23801
---------
Signed-off-by: Marvin Hagemeister <marvinhagemeister50@gmail.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
<!--
Before submitting a PR, please read
https://docs.deno.com/runtime/manual/references/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Fixes https://github.com/denoland/deno/issues/23397
Part of #22607 (probably closes it, but I haven't done thorough testing)
Makes it so that `require.resolve` with `paths` specified will fallback
to using the global cache when the paths can't be found when using a
global cache (not when using a node_modules folder)
This PR adds private `[REF]()` and `[UNREF]()` methods to Stdin class,
and call them from Node.js polyfill layer (`TTY` class). This enables
`process.stdin.unref()` and `process.stdin.ref()` for the case when
stdin is terminal.
closes #21796
When the response has been successfully send, we abort the
`Request.signal` property to indicate that all resources associated with
this transaction may be torn down.
Most common argument to `env` option for `worker_threads.Worker` will be
`process.env`.
In Deno `process.env` is a `Proxy` which can't be cloned using
structured clone algorithm.
So to be safe, I'm creating a copy of actual object before it's sent to
the worker thread.
Ref #23522
Landing work from #21903, plus fixing a node compat bug.
We were always sending the HTTP/2 ALPN on TLS connections which might
confuse upstream servers.
Changes:
- Configure HTTP/2 ALPN when making the TLS connection from the HTTP/2
code
- Read the `ALPNProtocols` property from the TLS connection options
rather than the deno `alpnProtocols` field
- Add tests
Prereq for landing Deno.serveHttp on Deno.serve: removing older HTTP
servers from the codebase.
Closes https://github.com/denoland/deno/issues/23362
Previously we were panicking if there was a pending read on a
port and `receiveMessageOnPort` was called. This is now fixed
by cancelling the pending read, trying to read a message and
resuming reading in a loop.
This PR follows this fix (https://github.com/nodejs/node/pull/52005) in
Node.js.
Stream's construct callback happens one tick earlier by this change, and
it prevents the reordering of the first few chunks in
`node:stream.Writable`
closes #20284
MessagePort if directly assigned to workerData property instead of
embedding it in an object then it is not patched to a NodeMessagePort.
This commit fixes the bug.
Fixes #23179.
Fixes #22454.
Enables passing `{tokens: true}` to `parseArgs` and setting default
values for options.
With this PR, the observable framework works with deno out of the box
(no unstable flags needed).
The existing code was basically copied straight from node, so this PR
mostly just updates that (out of date) vendored code. Also fixes some
issues with error exports (before this PR, in certain error cases we
were attempting to construct error classes that weren't actually in
scope).
The last change (in the second commit) adds a small hack so that we
actually exercise the `test-parse-args.js` node_compat test, previously
it was reported as passing though it should have failed. That test now
passes.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
The `tools/node_compat/node` submodule has been moved to
`tests/node_compat/runner/suite` and the remaining files within
`tools/node_compat` to `tests/node_compat/runner`.
Most of the changes are of the header within `tests/node_compat/test`
files. The `setup` and `test` tasks within `tests/node_comapt` execute
successfully.
Towards #22525
CC @mmastrac
