Improves #19100
Fixes #20356
Replaces #20428
Changes made in deno_core to support this:
- [x] Errors must be handled in setTimeout callbacks
- [x] Microtask ordering is not-quite-right
- [x] Timer cancellation must be checked right before dispatch
- [x] Timer sanitizer
- [x] Move high-res timer to deno_core
- [x] Timers need opcall tracing
fixes #22627
This PR fixes a node compat issue that is preventing `serverless-http`
and `serverless-express` npm modules from working with Deno. These
modules are useful for running apps on AWS Lambda (and other serverless
infra).
---------
Signed-off-by: Igor Zinkovsky <igor@deno.com>
Investigating https://github.com/denoland/deno/issues/22574
Unable to reproduce with a unit test, but assuming that it's a name
collision or create pipe/open pipe race, and adding some additional
diagnostics.
Gets us closer to solving #20707.
Rewrites the `TestEventSender`:
- Allow for explicit creation of multiple streams. This will allow for
one-std{out,err}-per-worker
- All test events are received along with a worker ID, allowing for
eventual, proper parallel threading of test events.
In theory this should open up proper interleaving of test output,
however that is left for a future PR.
I had some plans for a better performing synchronization primitive, but
the inter-thread communication is tricky. This does, however, speed up
the processing of large numbers of tests 15-25% (possibly even more on
100,000+).
Before
```
ok | 1000 passed | 0 failed (32ms)
ok | 10000 passed | 0 failed (276ms)
```
After
```
ok | 1000 passed | 0 failed (25ms)
ok | 10000 passed | 0 failed (230ms)
```
When using a prefix or suffix containing an invalid filename character,
it's not entirely clear where the errors come from. We make these errors
more consistent across platforms.
In addition, all permission prompts for tempfile and tempdir were
printing the same API name.
We also take the opportunity to make the tempfile random space larger by
2x (using a base32-encoded u64 rather than a hex-encoded u32).
This change deprecates
`Deno.CreateHttpClientOptions.{certChain,privateKey}` in favour of
`Deno.CreateHttpClientOptions.{cert,key}`.
Closes #22278
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
Splitting the sleep and interval ops allows us to detect an interval
timer. We also remove the use of the `op_async_void_deferred` call.
A future PR will be able to split the op sanitizer messages for timers
and intervals.
Based on #21074 and #20741 I was looking for further potential use cases
of `TransformStream` `cancel()` method, so here go `CompressionStream`
and `DecompressionStream`.
Fixes #14212
`certFile` was being passed to `op_tls_start()` when it's unused and
undocumented.
Signed-off-by: Asher Gomez <ashersaupingomez@gmail.com>
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
Note: tests are not the only part of the codebase that uses `std`. Other
parts, like `tools/`, do too. So, it could be argued that this is a
little misleading. Either way, I'm doing this as discussed with
@mmastrac.
This PR separates integration tests from CLI tests into a new project
named `cli_tests`. This is a prerequisite for an integration test runner
that can work with either the CLI binary in the current project, or one
that is built ahead of time.
## Background
Rust does not have the concept of artifact dependencies yet
(https://github.com/rust-lang/cargo/issues/9096). Because of this, the
only way we can ensure a binary is built before running associated tests
is by hanging tests off the crate with the binary itself.
Unfortunately this means that to run those tests, you _must_ build the
binary and in the case of the deno executable that might be a 10 minute
wait in release mode.
## Implementation
To allow for tests to run with and without the requirement that the
binary is up-to-date, we split the integration tests into a project of
their own. As these tests would not require the binary to build itself
before being run as-is, we add a stub integration `[[test]]` target in
the `cli` project that invokes these tests using `cargo test`.
The stub test runner we add has `harness = false` so that we can get
access to a `main` function. This `main` function's sole job is to
`execvp` the command `cargo test -p deno_cli`, effectively "calling"
another cargo target.
This ensures that the deno executable is always correctly rebuilt before
running the stub test runner from `cli`, and gets us closer to be able
to run the entire integration test suite on arbitrary deno executables
(and therefore split the build into multiple phases).
The new `cli_tests` project lives within `cli` to avoid a large PR. In
later PRs, the test data will be split from the `cli` project. As there
are a few thousand files, it'll be better to do this as a completely
separate PR to avoid noise.
This moves the op sanitizer descriptions into Rust code and prepares for
eventual op import from `ext:core/ops`. We cannot import these ops from
`ext:core/ops` as the testing infrastructure ops are not always present.
Changes:
- Op descriptions live in `cli` code and are currently accessible via an
op for the older sanitizer code
- `phf` dep moved to workspace root so we can use it here
- `ops.op_XXX` changed to to `op_XXX` to prepare for op imports later
on.
Migrations:
- Error registration no longer required for Interrupted or BadResource
(these are core exception)
- `include_js_files!`/`ExtensionFileSource` changes
This commit adds `ppid` getter for `node:process` to improve Node
compatibility one step further.
There is one problem though, which is that `Deno.ppid`, which
`process.ppid` internally calls, is actually of type `bigint` although
it's supposed to be `number`. I filed an issue for this (#22166). For
the time being, explciit type conversion from `bigint` to `number` is
applied to match the Node.js behavior.
Fixes https://github.com/denoland/deno/issues/22180
Matches the Node.js implementation more closely. Removed types, they do
not help just make it harder to debug with stack traces.
Follow up to #22157.
This leaves us with 4 usages of `ensureFastOps()` in `deno` itself.
There's also about 150 usages of `Deno.core.ops.<op_name>` left as well.
For removal in Deno v2. There are two issues:
1. Any script being run causes the output of `warnOnDeprecatedApi()` to
be printed, even when none of the `rid` properties are called.
2. `.rid` of these classes is used in multiple tests. I'm not sure how
to account for that. I thought of having `STDIN_RID`, and friends,
constants, whose values can be shared between the tests and the classes
themselves. Should we go with that or do something else?
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This change:
1. Implements `Deno.FsFile.sync()` and `Deno.FsFile.syncSync()`.
2. Deprecates `Deno.fsync()` and `Deno.fsyncSync()` for removal in Deno
v2, in favour of the above corresponding methods.
Related #21995
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This change:
1. Implements `Deno.FsFile.dataSync()` and `Deno.FsFile.dataSyncSync()`.
2. Deprecates `Deno.fdatasync()` and `Deno.fdatasyncSync()` for removal
in Deno v2, in favour of the above corresponding methods.
3. Replaces use of `Deno.fdatasync()` and `Deno.fdatasyncSync()` with
the above instance methods.
Related #21995
Most uses of `Deno.resources()` within tests, as they previously checked
for leaked resources. This is not needed as the test runner does this
automatically. Other internal uses of this API have been replaced with
the internal `Deno[Deno.internal].core.resources()`.
This change:
1. Implements `Deno.stdin.isTerminal()`, `Deno.stdout.isTerminal()` and
`Deno.stderr.isTerminal()`.
2. Deprecates `Deno.isatty()` for removal in Deno v2, in favour of the
above instance methods.
3. Replaces use of `Deno.isatty()` with the above instance methods.
Related #21995
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
This change:
1. Sets the removal version for `Deno.ListenTlsOptions.certFile`,
`Deno.ListenTlsOptions.keyFile` and `Deno.ConnectTlsOptions.certFile`
for Deno v2, in favour of the `cert`, `key` and `caCerts` options,
respectively.
2. Replaces use of the deprecated options with the new recommended
options.
Towards #22021
This commit deprecates "--unstable" flag.
When "--unstable" flag is encountered a warning like this is printed:
```
The `--unstable` flag is deprecated, use granular `--unstable-*` flags instead.
Learn more at: https://docs.deno.com/runtime/manual/tools/unstable_flags
```
When "--unstable" flag is used and an unstable API is called an
additional warning like this is printed for each API call:
```
The `Deno.dlopen` API was used with `--unstable` flag. The `--unstable` flag is deprecated, use granular `--unstable-ffi` instead.
Learn more at: https://docs.deno.com/runtime/manual/tools/unstable_flags
```
When no "--unstable-*" flag is provided and an unstable API is called
following
warning is issued before exiting:
```
Unstable API 'Deno.dlopen'. The `--unstable-ffi` flag must be provided.
```
---------
Signed-off-by: Divy Srivastava <dj.srivastava23@gmail.com>
Signed-off-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Co-authored-by: Asher Gomez <ashersaupingomez@gmail.com>
This change removes the currently deprecated `Deno.cron()` overload with
`options` as a potential last argument.
This might be fine to do now, in a major release, as `Deno.cron()` is an
unstable API. I thought of doing this while working on #22021. If this
is not ready to remove, I can instead set the removal version of this
overload for Deno v2.
Note: this overload was deprecated in Deno v1.38.2 (#21225). So it's
been deprecated for over 2 months.
Closes https://github.com/denoland/deno/issues/21828.
This API is a huge footgun. And given that "Deno.serveHttp" is a
deprecated API that is discouraged to use (use "Deno.serve()"
instead); it makes no sense to keep this API around.
This is a step towards fully migrating to Hyper 1.
<!--
Before submitting a PR, please read https://deno.com/manual/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
We were missing the `constants` export in the promise `fs` API which is
available in node.
```ts
import { constants, promises } from "node:fs";
import { constants as fsPromiseConstants } from "node:fs/promises";
console.log(constants === promises.constants); // logs: true
console.log(constants === fsPromiseConstants); // logs: true
```
Fixes https://github.com/denoland/deno/issues/21994
Introduces the first cppgc backed Resource into Deno.
This fixes the memory leak when using `X509Certificate`
**Comparison**:
```js
import { X509Certificate } from 'node:crypto';
const r = Deno.readFileSync('cli/tests/node_compat/test/fixtures/keys/agent1-cert.pem');
setInterval(() => {
for (let i = 0; i < 10000; i++) {
const cert = new X509Certificate(r);
}
}, 1000);
```
Memory usage after 5 secs
`main`: 1692MB
`cppgc`: peaks at 400MB
<!--
Before submitting a PR, please read https://deno.com/manual/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Ultimately, it came down to using incorrect property names in the
`FakeSocket` constructor. The values are passed under `remoteAddress`,
not `hostname` and `remotePort` instead of `port`.
Fixes https://github.com/denoland/deno/issues/21980
This commit adds support for [Stage 3 Temporal API
proposal](https://tc39.es/proposal-temporal/docs/).
The API is available when `--unstable-temporal` flag is passed.
---------
Signed-off-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: David Sherret <dsherret@gmail.com>
Co-authored-by: Kenta Moriuchi <moriken@kimamass.com>
This commit adds support for "rejectionhandled" Web Event and
"rejectionHandled" Node event.
```js
import process from "node:process";
process.on("rejectionHandled", (promise) => {
console.log("rejectionHandled", reason, promise);
});
window.addEventListener("rejectionhandled", (event) => {
console.log("rejectionhandled", event.reason, event.promise);
});
```
---------
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
<!--
Before submitting a PR, please read https://deno.com/manual/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
---------
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
Fixes https://github.com/denoland/deno/issues/21840
The problem was hard to reproduce as its a race condition. I've added a
test that reproduces the problem 1/10 tries. We should move the
idleTimeout handling to Rust (maybe even built into fastwebsocket).
<!--
Before submitting a PR, please read https://deno.com/manual/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Fixes https://github.com/denoland/deno/issues/21836
Part 1 of a potential 3 part series. Ref #13449
The current implementation passes key material back and forth RustCrypto
group of crates and ring. ring does not implement p521 yet.
This PR adds support for P521 named curve in `generateKey` and
`importKey` where we use RustCrypto. Other parts should be moved over to
the RustGroup group of crates for consistency.
Part 1 of #21713
Changes:
- Remove `.present()` and add a `presentGPUCanvasContext` (not exposed
yet to users)
- Move lazy load logic to `00_init.js`. This can be used to use webgpu
on-demand from future code (OffScreenCanvas)
Fixes #20516
Follow up to #21747 and #21746
This tackles the last point of #20516 where certain inputs weren't
accepted in the other zlib methods
This adds the `toU8` conversion of `_brotli` to `_zlib.mjs`, when we
create the ZLibBuffer, we'll sanitize the input. I noticed that the
async had no handler for `string` input so I added that as well.
Fixes https://github.com/denoland/deno/issues/21734
Changes:
- Use default encode when options do not provide a encode callback.
- Remove internal TS for `node:querystring`. Its not helping catching
bugs like this because of invalid type assumptions and use of `any`,
more of a maintenance burden.
This fixes point 3 of https://github.com/denoland/deno/issues/20516
This PR creates consistency between the sync and async versions of the
brotli compress where we will always return a buffer like Node.
This fixes point 2 of #20516
This adds a conversion from Dataview/Buffer by returning `obj.buffer`
which can be converted to a `UInt8Array`.
Question: Regarding point 4 of the mentioned issue would it be
appropriate to copy the toU8 helper to the `zlib.mjs` methods?
