This commit fixes race condition in "node:worker_threads" module were
the first message did a setup of "threadId", "workerData" and
"environmentData".
Now this data is passed explicitly during workers creation and is set up
before any user code is executed.
Closes https://github.com/denoland/deno/issues/22783
Closes https://github.com/denoland/deno/issues/22672
---------
Co-authored-by: Satya Rohith <me@satyarohith.com>
This commit changes how we figure out if we're running on main
thread in `node:worker_threads` module. Instead of relying on quirky
"magic variable" for a name to check if we're on main thread, we are
now explicitly passing this information during bootstrapping of the
runtime. As a side effect, `WorkerOptions.name` is more useful
and matches what Node.js does more closely (though not fully).
Towards https://github.com/denoland/deno/issues/22783
<!--
Before submitting a PR, please read
https://docs.deno.com/runtime/manual/references/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Improves #19100
Fixes #20356
Replaces #20428
Changes made in deno_core to support this:
- [x] Errors must be handled in setTimeout callbacks
- [x] Microtask ordering is not-quite-right
- [x] Timer cancellation must be checked right before dispatch
- [x] Timer sanitizer
- [x] Move high-res timer to deno_core
- [x] Timers need opcall tracing
- Removes the origin call, since all origins are the same for an isolate
(ie: the main module)
- Collects the `TestDescription`s and sends them all at the same time
inside of an Arc, allowing us to (later on) re-use these instead of
cloning.
Needs a follow-up pass to remove all the cloning, but that's a thread
that is pretty long to pull
---------
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
This commit removes some not really necessary FFI tests and in effect
removes them from being accessible from the user code.
This lowers the number of ops accessible to user code to 16.
This moves the op sanitizer descriptions into Rust code and prepares for
eventual op import from `ext:core/ops`. We cannot import these ops from
`ext:core/ops` as the testing infrastructure ops are not always present.
Changes:
- Op descriptions live in `cli` code and are currently accessible via an
op for the older sanitizer code
- `phf` dep moved to workspace root so we can use it here
- `ops.op_XXX` changed to to `op_XXX` to prepare for op imports later
on.
Migrations:
- Error registration no longer required for Interrupted or BadResource
(these are core exception)
- `include_js_files!`/`ExtensionFileSource` changes
This commit adds a list of ops to `runtime/99_main.js` that are
currently relying on getting them from `Deno.core.ops`. All ops that are not
present in the list are removed from `Deno.core.ops` on startup (they are
imported from "virtual op module" - `ext:core/ops`) making them effectively
inaccessible to user code.
This change lowers the number of ops exposed to user code from 650 to
around 260. This number should be gradually decreased in follow-up PRs.
Follow up to #22157.
This leaves us with 4 usages of `ensureFastOps()` in `deno` itself.
There's also about 150 usages of `Deno.core.ops.<op_name>` left as well.
For removal in Deno v2. I've also updated the deprecation of
`Deno.FsWatcher.return()`, which, to be clear, I'm not in favour of
deprecating. I mention this in #15499. Either way, it's safe to merge
this PR, then decide against the deprecation.
Removes weird "frame like" characters and simplifies the output:
```
warning: Use of deprecated "Deno.isatty()" API. This API will be removed in Deno 2.
Stack trace:
at file:///Users/ib/dev/deno/foo.js:2:8
hint: Use `stdStream.isTerminal()` instead.
warning: Use of deprecated "Deno.isatty()" API. This API will be removed in Deno 2.
Stack trace:
at file:///Users/ib/dev/deno/foo.js:7:8
hint: Use `stdStream.isTerminal()` instead.
```
https://github.com/denoland/deno/assets/13602871/7a6e24bf-44ec-4dbf-ac96-2af1db9f2ab9
This commit deprecates `window` global and adds deprecation
notice on each use of `window`.
We decided to proceed with removal of `window` global variable in Deno
2.0. There's a lot of code
in the wild that uses pattern like this:
```
if (typeof window !== "undefined) {
...
}
```
to check if the code is being run in browser. However, this check passes
fine in Deno and
most often libraries that do this check try to access some browser API
that is not available
in Deno, or use DOM APIs (which are also not available in Deno).
This situation has occurred multiple times already
and it's unfeasible to expect the whole ecosystem to migrate to new
check (and even if that
happened there's a ton of code that's already shipped and won't change).
The migration is straightfoward - replace all usages of `window` with
`globalThis` or `self`.
When Deno encounters use of `window` global it will now issue a warning,
steering users
towards required changes:
```
Warning
├ Use of deprecated "window" API.
│
├ This API will be removed in Deno 2.0. Make sure to upgrade to a stable API before then.
│
├ Suggestion: Use `globalThis` or `self` instead.
│
├ Suggestion: You can provide `window` in the current scope with: `const window = globalThis`.
│
└ Stack trace:
└─ at file:///Users/ib/dev/deno/foo.js:7:1
```
Ref https://github.com/denoland/deno/issues/13367.
Most uses of `Deno.resources()` within tests, as they previously checked
for leaked resources. This is not needed as the test runner does this
automatically. Other internal uses of this API have been replaced with
the internal `Deno[Deno.internal].core.resources()`.
This change:
1. Implements `Deno.stdin.isTerminal()`, `Deno.stdout.isTerminal()` and
`Deno.stderr.isTerminal()`.
2. Deprecates `Deno.isatty()` for removal in Deno v2, in favour of the
above instance methods.
3. Replaces use of `Deno.isatty()` with the above instance methods.
Related #21995
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
This change sets the removal version of `Deno.customInspect` for Deno
v2.
Towards #22021
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This change:
1. Sets the removal version for `Deno.RequestEvent`, `Deno.HttpConn` and
`Deno.serveHttp()` for Deno v2. I thought it might be worth calling
`warnOnDeprecatedApi()` within `Deno.Request` and `Deno.HttpConn`
methods, but I thought just having it called within `Deno.serveHttp()`
might be sufficient.
2. Removes some possibly unneeded related benchmarks.
Towards #22021
Closes https://github.com/denoland/deno/issues/21828.
This API is a huge footgun. And given that "Deno.serveHttp" is a
deprecated API that is discouraged to use (use "Deno.serve()"
instead); it makes no sense to keep this API around.
This is a step towards fully migrating to Hyper 1.
This commit introduces deprecation warnings for "Deno.*" APIs.
This is gonna be quite noisy, but should tremendously help with user
code updates to ensure
smooth migration to Deno 2.0. The warning is printed at each unique call
site to help quickly
identify where code needs to be adjusted. There's some stack frame
filtering going on to
remove frames that are not useful to the user and would only cause
confusion.
The warning can be silenced using "--quiet" flag or
"DENO_NO_DEPRECATION_WARNINGS" env var.
"Deno.run()" API is now using this warning. Other deprecated APIs will
start warning
in follow up PRs.
Example:
```js
import { runEcho as runEcho2 } from "http://localhost:4545/run/warn_on_deprecated_api/mod.ts";
const p = Deno.run({
cmd: [
Deno.execPath(),
"eval",
"console.log('hello world')",
],
});
await p.status();
p.close();
async function runEcho() {
const p = Deno.run({
cmd: [
Deno.execPath(),
"eval",
"console.log('hello world')",
],
});
await p.status();
p.close();
}
await runEcho();
await runEcho();
for (let i = 0; i < 10; i++) {
await runEcho();
}
await runEcho2();
```
```
$ deno run --allow-read foo.js
Warning
├ Use of deprecated "Deno.run()" API.
│
├ This API will be removed in Deno 2.0. Make sure to upgrade to a stable API before then.
│
├ Suggestion: Use "Deno.Command()" API instead.
│
└ Stack trace:
└─ at file:///Users/ib/dev/deno/foo.js:3:16
hello world
Warning
├ Use of deprecated "Deno.run()" API.
│
├ This API will be removed in Deno 2.0. Make sure to upgrade to a stable API before then.
│
├ Suggestion: Use "Deno.Command()" API instead.
│
└ Stack trace:
├─ at runEcho (file:///Users/ib/dev/deno/foo.js:8:18)
└─ at file:///Users/ib/dev/deno/foo.js:13:7
hello world
Warning
├ Use of deprecated "Deno.run()" API.
│
├ This API will be removed in Deno 2.0. Make sure to upgrade to a stable API before then.
│
├ Suggestion: Use "Deno.Command()" API instead.
│
└ Stack trace:
├─ at runEcho (file:///Users/ib/dev/deno/foo.js:8:18)
└─ at file:///Users/ib/dev/deno/foo.js:14:7
hello world
Warning
├ Use of deprecated "Deno.run()" API.
│
├ This API will be removed in Deno 2.0. Make sure to upgrade to a stable API before then.
│
├ Suggestion: Use "Deno.Command()" API instead.
│
└ Stack trace:
├─ at runEcho (file:///Users/ib/dev/deno/foo.js:8:18)
└─ at file:///Users/ib/dev/deno/foo.js:17:9
hello world
hello world
hello world
hello world
hello world
hello world
hello world
hello world
hello world
hello world
Warning
├ Use of deprecated "Deno.run()" API.
│
├ This API will be removed in Deno 2.0. Make sure to upgrade to a stable API before then.
│
├ Suggestion: Use "Deno.Command()" API instead.
│
├ Suggestion: It appears this API is used by a remote dependency.
│ Try upgrading to the latest version of that dependency.
│
└ Stack trace:
├─ at runEcho (http://localhost:4545/run/warn_on_deprecated_api/mod.ts:2:18)
└─ at file:///Users/ib/dev/deno/foo.js:20:7
hello world
```
Closes #21839
This commit adds support for [Stage 3 Temporal API
proposal](https://tc39.es/proposal-temporal/docs/).
The API is available when `--unstable-temporal` flag is passed.
---------
Signed-off-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: David Sherret <dsherret@gmail.com>
Co-authored-by: Kenta Moriuchi <moriken@kimamass.com>
This commit adds support for "rejectionhandled" Web Event and
"rejectionHandled" Node event.
```js
import process from "node:process";
process.on("rejectionHandled", (promise) => {
console.log("rejectionHandled", reason, promise);
});
window.addEventListener("rejectionhandled", (event) => {
console.log("rejectionhandled", event.reason, event.promise);
});
```
---------
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
Part 1 of #21713
Changes:
- Remove `.present()` and add a `presentGPUCanvasContext` (not exposed
yet to users)
- Move lazy load logic to `00_init.js`. This can be used to use webgpu
on-demand from future code (OffScreenCanvas)
`opAsync` requires a lookup by name on each async call. This is a
mechanical translation of all opAsync calls to ensureFastOps.
The `opAsync` API on Deno.core will be removed at a later time.
This PR implements the Node child_process IPC functionality in Deno on
Unix systems.
For `fd > 2` a duplex unix pipe is set up between the parent and child
processes. Currently implements data passing via the channel in the JSON
serialization format.
This fixes #21434 for `BroadcastChannel` and `WebSocketStream`.
`--unstable` still enable both, but granular unstable flags now also
work:
* `--unstable-net` now enables `WebSocketStream`.
* `--unstable-broadcast-channel` now enables `BroadcastChannel`.
* Additionally, there are now tests for all granular unstable flags.
Since `unsafe-proto` already had tests, so I didn't add any for this
one.
It also introduces a map to keep track of granular unstable ids without
having to sync multiple places.
This commit brings back usage of primordials in "40_testing.js" by
turning it back into an ES module and using new "lazy loading" functionality
of ES modules coming from "deno_core".
The same approach was applied to "40_jupyter.js".
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit refactors how we access "core", "internals" and
"primordials" objects coming from `deno_core`, in our internal JavaScript code.
Instead of capturing them from "globalThis.__bootstrap" namespace, we
import them from recently added "ext:core/mod.js" file.
Landing changes required for
https://github.com/denoland/deno_core/pull/359
We needed to update 99_main.js and a whole load of tests.
API changes:
- setPromiseRejectCallback becomes setUnhandledPromiseRejectionHandler.
The function is now called from eventLoopTick.
- The promiseRejectMacrotaskCallback no longer exists, as this is
automatically handled in eventLoopTick.
- ops.op_dispatch_exception now takes a second parameter: in_promise.
The preferred way to call this op is now reportUnhandledException or
reportUnhandledPromiseRejection.
We can move all promise ID knowledge to deno_core, allowing us to better
experiment with promise implementation in deno_core.
`{un,}refOpPromise(promise)` is equivalent to
`{un,}refOp(promise[promiseIdSymbol])`
This commit adds granular `--unstable-*` flags:
- "--unstable-broadcast-channel"
- "--unstable-ffi"
- "--unstable-fs"
- "--unstable-http"
- "--unstable-kv"
- "--unstable-net"
- "--unstable-worker-options"
- "--unstable-cron"
These flags are meant to replace a "catch-all" flag - "--unstable", that
gives a binary control whether unstable features are enabled or not. The
downside of this flag that allowing eg. Deno KV API also enables the FFI
API (though the latter is still gated with a permission).
These flags can also be specified in `deno.json` file under `unstable`
key.
Currently, "--unstable" flag works the same way - I will open a follow
up PR that will print a warning when using "--unstable" and suggest to use
concrete "--unstable-*" flag instead. We plan to phase out "--unstable"
completely in Deno 2.
This PR adds unstable `Deno.cron` API to trigger execution of cron jobs.
* State: All cron state is in memory. Cron jobs are scheduled according
to the cron schedule expression and the current time. No state is
persisted to disk.
* Time zone: Cron expressions specify time in UTC.
* Overlapping executions: not permitted. If the next scheduled execution
time occurs while the same cron job is still executing, the scheduled
execution is skipped.
* Retries: failed jobs are automatically retried until they succeed or
until retry threshold is reached. Retry policy can be optionally
specified using `options.backoffSchedule`.
To fix bugs around detection of when node emulation is required, we will
just eagerly initialize it. The improvements we make to reduce the
impact of the startup time:
- [x] Process stdin/stdout/stderr are lazily created
- [x] node.js global proxy no longer allocates on each access check
- [x] Process checks for `beforeExit` listeners before doing expensive
shutdown work
- [x] Process should avoid adding global event handlers until listeners
are added
Benchmarking this PR (`89de7e1ff`) vs main (`41cad2179`)
```
12:36 $ third_party/prebuilt/mac/hyperfine --warmup 100 -S none './deno-41cad2179 run ./empty.js' './deno-89de7e1ff run ./empty.js'
Benchmark 1: ./deno-41cad2179 run ./empty.js
Time (mean ± σ): 24.3 ms ± 1.6 ms [User: 16.2 ms, System: 6.0 ms]
Range (min … max): 21.1 ms … 29.1 ms 115 runs
Benchmark 2: ./deno-89de7e1ff run ./empty.js
Time (mean ± σ): 24.0 ms ± 1.4 ms [User: 16.3 ms, System: 5.6 ms]
Range (min … max): 21.3 ms … 28.6 ms 126 runs
```
Fixes https://github.com/denoland/deno/issues/20142
Fixes https://github.com/denoland/deno/issues/15826
Fixes https://github.com/denoland/deno/issues/20028
This commit adds new "--deny-*" permission flags. These are complimentary to
"--allow-*" flags.
These flags can be used to restrict access to certain resources, even if they
were granted using "--allow-*" flags or the "--allow-all" ("-A") flag.
Eg. specifying "--allow-read --deny-read" will result in a permission error,
while "--allow-read --deny-read=/etc" will allow read access to all FS but the
"/etc" directory.
Runtime permissions APIs ("Deno.permissions") were adjusted as well, mainly
by adding, a new "PermissionStatus.partial" field. This field denotes that
while permission might be granted to requested resource, it's only partial (ie.
a "--deny-*" flag was specified that excludes some of the requested resources).
Eg. specifying "--allow-read=foo/ --deny-read=foo/bar" and then querying for
permissions like "Deno.permissions.query({ name: "read", path: "foo/" })"
will return "PermissionStatus { state: "granted", onchange: null, partial: true }",
denoting that some of the subpaths don't have read access.
Closes #18804.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Nayeem Rahman <nayeemrmn99@gmail.com>
This addresses issue #19918.
## Issue description
Event messages have the wrong isTrusted value when they are not
triggered by user interaction, which differs from the browser. In
particular, all MessageEvents created by Deno have isTrusted set to
false, even though it should be true.
This is my first ever contribution to Deno, so I might be missing
something.
Code run within Deno-mode and Node-mode should have access to a
slightly different set of globals. Previously this was done through a
compile time code-transform for Node-mode, but this is not ideal and has
many edge cases, for example Node's globalThis having a different
identity than Deno's globalThis.
This commit makes the `globalThis` of the entire runtime a semi-proxy.
This proxy returns a different set of globals depending on the caller's
mode. This is not a full proxy, because it is shadowed by "real"
properties on globalThis. This is done to avoid the overhead of a full
proxy for all globalThis operations.
The globals between Deno-mode and Node-mode are now properly segregated.
This means that code running in Deno-mode will not have access to Node's
globals, and vice versa. Deleting a managed global in Deno-mode will
NOT delete the corresponding global in Node-mode, and vice versa.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Aapo Alasuutari <aapo.alasuutari@gmail.com>
This commit fixes emitting "unhandledrejection" event when there are
"node:" or "npm:" imports.
Before this commit the Node "unhandledRejection" event was emitted
using a regular listener for Web "unhandledrejection" event. This
listener was installed before any user listener had a chance to be
installed which effectively prevent emitting "unhandledrejection"
events to user code.
Closes https://github.com/denoland/deno/issues/16928
1. Rewrites the tests to be more back and forth rather than getting the
output all at once (which I believe was causing the hangs on linux and
maybe mac)
2. Runs the pty tests on the linux ci.
3. Fixes a bunch of tests that were just wrong.
4. Adds timeouts on the pty tests.
This commit changes how data required to bootstrap main and worker
runtime is serialized.
Instead of relying on serde_v8 and using JSON object,
we're doing manual serialization to a "v8::Array". This limits number
of V8 strings that need to be serialized by 16.
It also made it clear that some data could be obtained during
snapshotting instead of during bootstrap.
Since we can preserve ops in the snapshot these days, we no longer
need to have "Deno[Deno.internal].nodeUnstable" namespace.
Instead, various built-in Node.js modules can use appropriate APIs
directly.
This commit adds unstable "Deno.openKv()" API that allows to open
a key-value database at a specified path.
---------
Co-authored-by: Luca Casonato <hello@lcas.dev>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This PR _**temporarily**_ removes WebGPU (which has behind the
`--unstable` flag in Deno), due to performance complications due to its
presence.
It will be brought back in the future; as a point of reference, Chrome
will ship WebGPU to stable on 26/04/2023.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit changes how "unload" event is handled - before
this commit an event listener was added unconditionally in
the runtime bootstrapping function, which for some reason was
very expensive (0.3ms). Instead of adding an event listener,
a check was added to "dispatchEvent" function that performs
the same action (so it's only called if there's an event dispatched).
This commit renames "deno_core::InternalModuleLoader" to
"ExtModuleLoader" and changes the specifiers used by the
modules loaded from this loader to "ext:".
"internal:" scheme was really ambiguous and it's more characters than
"ext:", which should result in slightly smaller snapshot size.
Closes https://github.com/denoland/deno/issues/18020
This commit moves some of the code from "99_main.js" to
be executed during the snapshot time instead of on each
worker bootstrap. These should minimally help with startup
time benchmark.
This API is required by several extensions like "ext/node", "ext/ffi"
and also FS APIs that we want to move to a separate crate. Because
of that "pathFromURL" API was moved to "deno_web" extension so
other extension crates can rely on it.
We use information about build in several extension crates like
"ext/node" or "runtime/". In an effort to move "fs" APIs to a separate
crate it is a prerequisite to have this information available outside
of the "runtime/" crate.
This commit moves definition of "build" object to "Deno.core" that is
later forwarded to "Deno.build".
This commit merges "runtime/js/40_spawn.js" into
"runtime/js/40_process.js", and "runtime::ops::spawn"
into "runtime::ops::process".
It makes little sense to have them separated given that we want to
factor out these APIs into a separate extension crate.
JavaScript APIs from "runtime/js/40_files.js" combined abstractions
for stdio streams ("Stdout", "Stderr", "Stdin") and file system file
("File", "FsFile"). APIs from "runtime/js/40_read_file.js" and
"runtime/js/40_write_file.js" were implemented using ops from
"runtime/ops/fs.rs".
This file was removed and relevant APIs were moved to "deno_io/12_io.js"
and "runtime/js/30_fs.js".
This work is meant to enable factoring out "deno_fs" crate.
This file was not used anywhere - even though it exposed an unstable
"Deno.DiagnosticCategory" API. Seems like a left over from when we
had "Deno.emit" API.
