This PR implements the child_process IPC pipe between parent and child.
The implementation uses Windows named pipes created by parent and passes
the inheritable file handle to the child.
I've also replace parts of the initial implementation which passed the
raw parent fd to JS with resource ids instead. This way no file handle
is exposed to the JS land (both parent and child).
`IpcJsonStreamResource` can stream upto 800MB/s of JSON data on Win 11
AMD Ryzen 7 16GB (without `memchr` vectorization)
`napi_open_handle_scope` was returning a bogus handle_scope and we were
trying to close it in `napi_close_handle_scope`.
This is a bit of a challenge to test, but the following testcase comes
from #21601 and appears to be fixed by this.
```
import { decode } from "https://deno.land/std@0.209.0/encoding/base64.ts";
import sharp from "npm:sharp";
const img = 'iVBORw0KGgoAAAANSUhEUgAAAQAAAAEAAQMAAABmvDolAAAAA1BMVEWq09/P7Lz1AAAAH0lEQVRoge3BAQ0AAADCoPdPbQ43oAAAAAAAAAAAvg0hAAABmmDh1QAAAABJRU5ErkJggg==';
Deno.test("async", async () => {
const id = setTimeout(() => Deno.exit(1), 1000);
await sharp(decode(img)).toBuffer();
await sharp(decode(img)).toBuffer();
clearTimeout(id);
});
```
Fixes https://github.com/denoland/deno/issues/21594
I verified locally that this fixes the problem. I'm working on testing
harness for Jupyter kernel to catch regressions like this and will
add it in a follow up PR.
Bumped versions for 1.39.0
Please ensure:
- [x] Target branch is correct (`vX.XX` if a patch release, `main` if
minor)
- [x] Crate versions are bumped correctly
- [x] deno_std version is incremented in the code (see
`cli/deno_std.rs`)
- [x] Releases.md is updated correctly (think relevancy and remove
reverts)
To make edits to this PR:
```shell
git fetch upstream release_1_39.0 && git checkout -b release_1_39.0 upstream/release_1_39.0
```
cc @mmastrac
---------
Co-authored-by: mmastrac <mmastrac@users.noreply.github.com>
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
This change deprecates `Deno.Reader`, `Deno.ReaderSync`, `Deno.Writer`,
`Deno.WriterSync` and `Deno.Closer` in favour of the [Web Streams
API](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API).
After discussing with Yoshiya, we both thought now might be the right
time to deprecate these interfaces with v2 getting closer.
This PR implements the Node child_process IPC functionality in Deno on
Unix systems.
For `fd > 2` a duplex unix pipe is set up between the parent and child
processes. Currently implements data passing via the channel in the JSON
serialization format.
Part 2 of removing middleware.
This is somewhat awkward because `V8CrossThreadTaskSpawner` requires
tasks to be `Send`, but NAPI makes heavy use of `!Send` pointers. In
addition, Rust causes a closure to be `!Send` if you pull a `!Send`
value out of a struct.
---------
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Adds performance measurements for all ops used by the LSP. Also changes
output of "Language server status" page to include more precise
information.
Current suspicion is that computing "script version" takes a long time
for some users.
This problem occurred trying to load tensorflow.js
```
> import * as tf from 'npm:@tensorflow/tfjs';
Uncaught SyntaxError: Identifier 'mod' has already been declared at file:///Users/ry/Library/Caches/deno/npm/registry.npmjs.org/@tensorflow/tfjs/4.14.0/dist/tf.node.js:167:14
at async <anonymous>:1:33
```
This commit adds a no-op flushHeaders method to the ServerResponse
object. It is a nop because the ServerResponse implementation is based
on top of the Deno server API instead of the Node `OutgoingMessage`
base.
Fixes #21509
This fixes #21434 for `BroadcastChannel` and `WebSocketStream`.
`--unstable` still enable both, but granular unstable flags now also
work:
* `--unstable-net` now enables `WebSocketStream`.
* `--unstable-broadcast-channel` now enables `BroadcastChannel`.
* Additionally, there are now tests for all granular unstable flags.
Since `unsafe-proto` already had tests, so I didn't add any for this
one.
It also introduces a map to keep track of granular unstable ids without
having to sync multiple places.
This commit brings back usage of primordials in "40_testing.js" by
turning it back into an ES module and using new "lazy loading" functionality
of ES modules coming from "deno_core".
The same approach was applied to "40_jupyter.js".
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Adds an `--unstable-sloppy-imports` flag which supports the
following for `file:` specifiers:
* Allows writing `./mod` in a specifier to do extension probing.
- ex. `import { Example } from "./example"` instead of `import { Example
} from "./example.ts"`
* Allows writing `./routes` to do directory extension probing for files
like `./routes/index.ts`
* Allows writing `./mod.js` for *mod.ts* files.
This functionality is **NOT RECOMMENDED** for general use with Deno:
1. It's not as optimal for perf:
https://marvinh.dev/blog/speeding-up-javascript-ecosystem-part-2/
1. It makes tooling in the ecosystem more complex in order to have to
understand this.
1. The "Deno way" is to be explicit about what you're doing. It's better
in the long run.
1. It doesn't work if published to the Deno registry because doing stuff
like extension probing with remote specifiers would be incredibly slow.
This is instead only recommended to help with migrating existing
projects to Deno. For example, it's very useful for getting CJS projects
written with import/export declaration working in Deno without modifying
module specifiers and for supporting TS ESM projects written with
`./mod.js` specifiers.
This feature will output warnings to guide the user towards correcting
their specifiers. Additionally, quick fixes are provided in the LSP to
update these specifiers:
Landing changes required for
https://github.com/denoland/deno_core/pull/359
We needed to update 99_main.js and a whole load of tests.
API changes:
- setPromiseRejectCallback becomes setUnhandledPromiseRejectionHandler.
The function is now called from eventLoopTick.
- The promiseRejectMacrotaskCallback no longer exists, as this is
automatically handled in eventLoopTick.
- ops.op_dispatch_exception now takes a second parameter: in_promise.
The preferred way to call this op is now reportUnhandledException or
reportUnhandledPromiseRejection.
This commit stabilizes "Deno.HttpServer.shutdown" API as well as
Unix socket support in "Deno.serve" API.
---------
Co-authored-by: Yoshiya Hinosawa <stibium121@gmail.com>
A bit hacky, but it works. Essentially, this will check for all the
scripts in the node_modules/.bin directory then force them to run with
Deno via deno_task_shell.
This commit adds a method of `Symbol.dispose` to the object returned
from `Deno.createHttpClient`, so we can make use of [explicit resource
management](https://github.com/tc39/proposal-explicit-resource-management)
by declaring it with `using`.
This commit adds support for a new `kv.watch()` method that allows
watching for changes to a key-value pair. This is useful for cases
where you want to be notified when a key-value pair changes, but
don't want to have to poll for changes.
---------
Co-authored-by: losfair <zhy20000919@hotmail.com>
Add support for signing with a RSA PEM private key: `pkcs8` and `pkcs1`.
Fixes https://github.com/denoland/deno/issues/18972
Ref #21124
Verified fix with `npm:sshpk`. Unverfied but fixes
`npm:google-auth-library`, `npm:web-push` & `oracle/oci-typescript-sdk`
---------
Signed-off-by: Divy Srivastava <dj.srivastava23@gmail.com>
This commit removes some of the technical debt related
to snapshotting JS code:
- "cli/ops/mod.rs" and "cli/build.rs" no longer define "cli" extension
which was not required anymore
- Cargo features for "deno_runtime" crate have been unified in
"cli/Cargo.toml"
- "cli/build.rs" uses "deno_runtime::snapshot::create_runtime_snapshot"
API
instead of copy-pasting the code
- "cli/js/99_main.js" was completely removed as it's not necessary
anymore
Towards https://github.com/denoland/deno/issues/21137
This PR causes Deno to include more files in the graph based on how a
template literal looks that's provided to a dynamic import:
```ts
const file = await import(`./dir/${expr}`);
```
In this case, it will search the `dir` directory and descendant
directories for any .js/jsx/etc modules and include them in the graph.
To opt out of this behaviour, move the template literal to a separate
line:
```ts
const specifier = `./dir/${expr}`
const file = await import(specifier);
```
Switch `ext/fetch` over to `resourceForReadableStream` to simplify and
unify implementation with `ext/serve`. This allows us to work in Rust
with resources only.
Two additional changes made to `resourceForReadableStream` were
required:
- Add an optional length to `resourceForReadableStream` which translates
to `size_hint`
- Fix a bug where writing to a closed stream that was full would panic
This commit changes LSP log names by prefixing them, we now have these
prefixes:
- `lsp.*` - requests coming from the client
- `tsc.request.*` - requests coming from clients that are routed to TSC
- `tsc.op.*` - ops called by the TS host
- `tsc.host.*` - requests that call JavaScript runtime that runs
TypeScript compiler host
Additionall `Performance::mark` was split into `Performance::mark` and
`Performance::mark_with_args` to reduce verbosity of code and logs.
This commit removes "ThrottledCancellationToken" in favor of
"CancellationToken".
Since calling into Rust to check if Tokio's cancellation token has
already been canceled is really cheap, there's no need for us to
throttle this check and let TSC burn up CPU with heavy computation.
When an old request is obsoleted while the user is typing, the client
will say so to the server and tower-lsp will drop the future associated
with that request.
This wires that up to the ts server by having any request's token be
cancelled when the surrounding state is dropped.
The self-upgrade feature is undesirable when deno is installed from
(Linux) distribution repository - using a system package manager. This
change will allow package maintainers to build deno with the "upgrade"
subcommand and background check disabled.
When the user runs `deno upgrade <args>` and the upgrade feature is
disabled, it will exit with error message explaining that this deno
binary was built without the upgrade feature.
Note: This patch is already used in the Alpine Linux’s
[deno](https://pkgs.alpinelinux.org/packages?name=deno) package.
What `Deno.ChildProcess` actually implements is `AsyncDisposable`, but the type
declaration says it's `Disposable`. This PR fixes the type declaration to match
the actual implementation.
Issue was main does canary builds, which broke this test because it
didn't handle searching for a canary release. Tested by building as
canary locally.
This PR updates the deprecation notices to point to the same replacement
APIs that the Standard Library points to. I've also tweaked the notices
to be a little more presentable/navigatable.
In particular, a follow-up PR in std will be made that documents the use
of `toArrayBuffer()`.
Closes #21193
Towards #20976
This commit adds unstable workspace support. This is extremely
bare-bones and
minimal first-pass at this.
With this change `deno.json` supports specifying `workspaces` key, that
accepts a list of subdirectories. Each workspace can have its own import
map. It's required to specify a `"name"` and `"version"` properties in the
configuration file for the workspace:
```jsonc
// deno.json
{
"workspaces": [
"a",
"b"
},
"imports": {
"express": "npm:express@5"
}
}
```
``` jsonc
// a/deno.json
{
"name": "a",
"version": "1.0.2",
"imports": {
"kleur": "npm:kleur"
}
}
```
```jsonc
// b/deno.json
{
"name": "b",
"version": "0.51.0",
"imports": {
"chalk": "npm:chalk"
}
}
```
`--unstable-workspaces` flag is required to use this feature:
```
$ deno run --unstable-workspaces mod.ts
```
---------
Co-authored-by: David Sherret <dsherret@gmail.com>
This PR changes the `Deno.cron` API:
* Marks the existing function as deprecated
* Introduces 2 new overloads, where the handler arg is always last:
```ts
Deno.cron(
name: string,
schedule: string,
handler: () => Promise<void> | void,
)
Deno.cron(
name: string,
schedule: string,
options?: { backoffSchedule?: number[]; signal?: AbortSignal },
handler: () => Promise<void> | void,
)
```
This PR also fixes a bug, when other crons continue execution after one
of the crons was closed using `signal`.
Fixes #21121 and #19498
Migrates fully to rustls_tokio_stream. We no longer need to maintain our
own TlsStream implementation to properly support duplex.
This should fix a number of errors with TLS and websockets, HTTP and
"other" places where it's failing.
This PR uses the new `cancel` method of `TransformStream` to properly
clean up the internal `TextDecoder` used in `TextDecoderStream` if the
stream is cancelled.
Fixes #13142
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
We only want one zlib dependency.
Zlib dependencies are reorganized so they use a hidden
`__vendored_zlib_ng` flag in cli that enables zlib-ng for both libz-sys
(used by ext/node) and flate2 (used by deno_web).
This also updates deno_graph, which has the JSR change to use "exports".
It's not yet useful atm, so I've made this PR a fix about the deno doc
--lint error message improvements. I'll do a follow-up PR that adds
exports to the deno.json
This is the release commit being forwarded back to main for 1.38.1
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Co-authored-by: littledivy <littledivy@users.noreply.github.com>
This PR reduces the time to run `queue persistence with inflight
messages` test from 60 seconds down to about 6 seconds.
The test simulates a crash to ensure that inflight messages are cleaned
up and re-queued when the new process starts. Since messages are
considered dead after 5 seconds of being queued, reopening the db within
5 seconds does not re-queue the messages on startup (they do get
re-queued after 60 seconds, which is the cleanup frequency). By waiting
for 5 seconds before reopening the db, the test ensures that the cleanup
happens quickly when the db is opened.
We can move all promise ID knowledge to deno_core, allowing us to better
experiment with promise implementation in deno_core.
`{un,}refOpPromise(promise)` is equivalent to
`{un,}refOp(promise[promiseIdSymbol])`
Remove tokio-rustls as a direct dependency of Deno and refactor
test_server to reduce code duplication.
All tcp and tls listener paths go through the same streams now, with the
exception of the simpler Hyper http-only handlers (those can be done in
a later follow-up).
Minor bugs fixed:
- gRPC server should only serve h2
- WebSocket over http/2 had a port overlap
- Restored missing eye-catchers for some servers (still missing on Hyper
ones)
