For the following example, if I set the encoding to `base64url`, it'll
throw an unexpected TypeError:
```ts
import { Buffer } from "node:buffer";
Buffer.from("IntcImhlbGxvXCI6XCJoZGQvZStpXCJ9Ig", "base64url").toString();
// error: Uncaught TypeError: src.subarray is not a function
// const buf = Buffer.from(
// ^
// at blitBuffer (ext:deno_node/internal/buffer.mjs:1779:15)
// at Uint8Array.base64urlWrite (ext:deno_node/internal/buffer.mjs:691:10)
// at Object.write (ext:deno_node/internal/buffer.mjs:2195:11)
// at Uint8Array.write (ext:deno_node/internal/buffer.mjs:794:14)
// at fromString (ext:deno_node/internal/buffer.mjs:214:22)
// at _from (ext:deno_node/internal/buffer.mjs:119:12)
// at Function.from (ext:deno_node/internal/buffer.mjs:157:10)
// at file:///Users/foodieats/temp/buffer1.ts:3:20
```
The error caused by `base64urlWrite` function, it should call
`forgivingBase64UrlDecode` not `forgivingBase64UrlEncode`
Also fixed #20563 .
This commit adds "deno jupyter" subcommand which
provides a Deno kernel for Jupyter notebooks.
The implementation is mostly based on Deno's REPL and
reuses large parts of it (though there's some clean up that
needs to happen in follow up PRs). Not all functionality of
Jupyter kernel is implemented and some message type
are still not implemented (eg. "inspect_request") but
the kernel is fully working and provides all the capatibilities
that the Deno REPL has; including TypeScript transpilation
and npm packages support.
Closes https://github.com/denoland/deno/issues/13016
---------
Co-authored-by: Adam Powers <apowers@ato.ms>
Co-authored-by: Kyle Kelley <rgbkrk@gmail.com>
This commit improves compatibility of "node:http2" module by polyfilling
"connect" method and "ClientHttp2Session" class. Basic operations like
streaming, header and trailer handling are working correctly.
Refing/unrefing is still a TODO and "npm:grpc-js/grpc" is not yet working
correctly.
---------
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
Fixes https://github.com/denoland/deno/issues/19816
In that issue, I suggest switching over the other brotli functionality
to the Rust API provided by the `brotli` crate. Here, I only do that
with the `brotli_decompress` function to fix the bug with buffers longer
than 4096 bytes.
When trying to run
```
deno run -A --unstable npm:astro dev
```
in my Astro project it fails with:
```
Node.js v18.12.1 is not supported by Astro!
Please upgrade Node.js to a supported version: ">=18.14.1"
```
My current version is:
```
~ ❯ node --version
v20.5.1
```
Bumping the version to the latest stable Release of node in
`ext/node/polyfills/_process/process.ts` fixes this.
I don't know if this causes any conflicts, so please feel free to
correct me here.
The fix for #20188 was not entirely correct -- we were unlocking the
global buffer incorrectly. This PR introduces a lock state that ensures
we only unlock a lock we have taken out.
<!--
Before submitting a PR, please read https://deno.com/manual/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
As the title.
---------
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
The goal of this PR is to address issue #20106 where a `TypeError`
occurs when the variables `uid` and `gid` from `userInfo()` in `node:os`
are reassigned if the user is on Windows. Both `uid` and `gid` are
marked as `const` therefore producing a `TypeError` when the two are
reassigned.
This PR achieves that goal by marking `uid` and `gid` as `let`
To fix bugs around detection of when node emulation is required, we will
just eagerly initialize it. The improvements we make to reduce the
impact of the startup time:
- [x] Process stdin/stdout/stderr are lazily created
- [x] node.js global proxy no longer allocates on each access check
- [x] Process checks for `beforeExit` listeners before doing expensive
shutdown work
- [x] Process should avoid adding global event handlers until listeners
are added
Benchmarking this PR (`89de7e1ff`) vs main (`41cad2179`)
```
12:36 $ third_party/prebuilt/mac/hyperfine --warmup 100 -S none './deno-41cad2179 run ./empty.js' './deno-89de7e1ff run ./empty.js'
Benchmark 1: ./deno-41cad2179 run ./empty.js
Time (mean ± σ): 24.3 ms ± 1.6 ms [User: 16.2 ms, System: 6.0 ms]
Range (min … max): 21.1 ms … 29.1 ms 115 runs
Benchmark 2: ./deno-89de7e1ff run ./empty.js
Time (mean ± σ): 24.0 ms ± 1.4 ms [User: 16.3 ms, System: 5.6 ms]
Range (min … max): 21.3 ms … 28.6 ms 126 runs
```
Fixes https://github.com/denoland/deno/issues/20142
Fixes https://github.com/denoland/deno/issues/15826
Fixes https://github.com/denoland/deno/issues/20028
Rename some of the helper methods on the Fs trait to be suffixed with
`_sync` / `_async`, in preparation of the introduction of more async
methods for some helpers.
Also adds a `read_text_file_async` helper to complement the renamed
`read_text_file_sync` helper.
Closes #19399 (running without snapshots at all was suggested as an
alternative solution).
Adds a `__runtime_js_sources` pseudo-private feature to load extension
JS sources at runtime for faster development, instead of building and
loading snapshots or embedding sources in the binary. Will only work in
a development environment obviously.
Try running `cargo test --features __runtime_js_sources
integration::node_unit_tests::os_test`. Then break some behaviour in
`ext/node/polyfills/os.ts` e.g. make `function cpus() {}` return an
empty array, and run it again. Fix and then run again. No more build
time in between.
This is a prerequisite for fast streams work -- this particular resource
used a custom `mpsc`-style stream, and this work will allow us to unify
it with the streams in `ext/http` in time.
Instead of using Option as an internal semaphore for "correctly
completed EOF", we allow code to propagate errors into the channel which
can be picked up by downstream sinks like Hyper. EOF is signalled using
a more standard sender drop.
This commit provides basic polyfill for "node:test" module. Currently
only top-level "test" function is polyfilled, all remaining functions from
that module throw not implemented errors.
V8 doesn't like having internal slots on the "real" globalThis object.
This commit works around this limitation by storing the inner globalThis
objects for segregated globals in a context slot.
Code run within Deno-mode and Node-mode should have access to a
slightly different set of globals. Previously this was done through a
compile time code-transform for Node-mode, but this is not ideal and has
many edge cases, for example Node's globalThis having a different
identity than Deno's globalThis.
This commit makes the `globalThis` of the entire runtime a semi-proxy.
This proxy returns a different set of globals depending on the caller's
mode. This is not a full proxy, because it is shadowed by "real"
properties on globalThis. This is done to avoid the overhead of a full
proxy for all globalThis operations.
The globals between Deno-mode and Node-mode are now properly segregated.
This means that code running in Deno-mode will not have access to Node's
globals, and vice versa. Deleting a managed global in Deno-mode will
NOT delete the corresponding global in Node-mode, and vice versa.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Aapo Alasuutari <aapo.alasuutari@gmail.com>
This makes the implementation of "AsyncLocalStorage" from
"node:async_hooks" 3.5x faster than before for noop benchmark
(measuring baseline overhead). It's still 3.5x slower than not
using `AsyncLocalStorage` and 1.64x slower than using
noop promise hooks.
Fixes #19568
Values are not coerced to the desired type during deserialisation. This
makes serde_v8 stricter.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
`ZeroCopyBuf` was convenient to use, but sometimes it did hide details
that some copies were necessary in certain cases. Also it made it way to easy
for the caller to pass around and convert into different values. This commit
splits `ZeroCopyBuf` into `JsBuffer` (an array buffer coming from V8) and
`ToJsBuffer` (a Rust buffer that will be converted into a V8 array buffer).
As a result some magical conversions were removed (they were never used)
limiting the API surface and preparing for changes in #19534.
This commit adds basic support for "node:http2" module. Not
all APIs have been yet implemented, but this change already
allows to use this module for some basic functions.
The "grpc" package is still not working, but it's a good stepping
stone.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
## WHY
ref: https://github.com/denoland/deno/issues/19165
The FileHandle class has many missing methods compared to node.
Add these.
## WHAT
- Add close method
---------
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
<!--
Before submitting a PR, please read https://deno.com/manual/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Internally, `node-tap` spawns a child process with `stdio: [0, 1, 2]`.
Whilst we don't support passing fd numbers as an argument so far, it
turns out that `[0, 1, 2]` is equivalent to `"inherit"` which we already
support. See: https://nodejs.org/api/child_process.html#optionsstdio
Mapping it to `"inherit"` is fine for us and gets us one step closer in
getting `node-tap` working. I'm now at the stage where already the
coverage table is shown 🎉
<!--
Before submitting a PR, please read https://deno.com/manual/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
## WHY
ref: https://github.com/denoland/deno/issues/19165
Node's fs/promises includes a FileHandle class, but deno does not. The
open function in Node's fs/promises returns a FileHandle, which provides
an IO interface to the file. However, deno's open function returns a
resource id.
### deno
```js
> const fs = await import("node:fs/promises");
undefined
> const file3 = await fs.open("./README.md");
undefined
> file3
3
> file3.read
undefined
Node:
```
### Node
```js
> const fs = await import("fs/promises");
undefined
> const file3 = await fs.open("./tests/e2e_unit/testdata/file.txt");
undefined
> file3
FileHandle {
_events: [Object: null prototype] {},
_eventsCount: 0,
_maxListeners: undefined,
close: [Function: close],
[Symbol(kCapture)]: false,
[Symbol(kHandle)]: FileHandle {},
[Symbol(kFd)]: 24,
[Symbol(kRefs)]: 1,
[Symbol(kClosePromise)]: null
}
> file3.read
[Function: read]
```
To be compatible with Node, deno's open function should also return a
FileHandle.
## WHAT
I have implemented the first step in adding a FileHandle.
- Changed the return value of the open function to a FileHandle object
- Implemented the readFile method in FileHandle
- Add test code
## What to do next
This PR is the first step in adding a FileHandle, and there are things
that should be done next.
- Add functionality equivalent to Node's FileHandle to FileHandle
(currently there is only readFile)
---------
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
This PR adds the missing `process.reallyExit()` method to node's
`process` object.
Was [pinged on
twitter](https://twitter.com/biwanczuk/status/1663326659787862017)
regarding running the `fastify` test suite in node. They use `node-tap`
which has been around arguably the longest of the test frameworks and
relies on a couple of old APIs. They have `signal-exit` as a dependency
which in turn [makes use of
`process.reallyExit()`](8fa7fc9a9c/src/index.ts (L19)).
That function cannot be found anywhere in their documentation, but
exists at runtime. See
6a6b3c5402/lib/internal/bootstrap/node.js (L172)
This doesn't yet make `node-tap` work, but gets us one step closer.
Rather than disallowing `ext:` resolution, clear the module map after
initializing extensions so extension modules are anonymized. This
operation is explicitly called in `deno_runtime`. Re-inject `node:`
specifiers into the module map after doing this.
Fixes #17717.
