Closes #19697. This fixes a bug where the writeFile API can create
partially-overwritten files which may lead to invalid / corrupt files or
data leakage. It also aligns the behavior of writing a ReadableStream
and writing a Uint8Array to the disk.
`createAsyncFromSyncIterator(x)` which is used in
`ReadableStream.from()` expects `x` as `Iterable` but, previous
implements specify `Iterator` or `IterableIterator`. If it was
`IterableIterator`, it would work, but if it was `Iterator`, an
exception will occur.
Tests have been merged into WPT.
https://github.com/web-platform-tests/wpt/pull/46365
---------
Co-authored-by: Asher Gomez <ashersaupingomez@gmail.com>
Enhanced warning message for --env flag with run and eval subcommands.
The commit is specifically made to address issue #23674 by improving the
warning messages that appear when using the --env flag with run or eval
subcommands in the following scenarios:
1. Missing environment file.
2. Incorrect syntax in the environment file content.
**Changes made**
- Distinguishes between cases of missing environment file and wrong
syntax in the environment file content.
- Shows a concise warning message to convey the case/issue occurred.
**Code changes & enhancements**
- Implemented a match statement to handle different types of errors
received while getting and parsing the file content to display a concise
warning message, rather than simple error check and then displaying the
same warning message for whatever the type of error is.
- Updated the related existing tests to reflect the new warning
messages.
- Added two test cases to cover the wrong environment file content
syntax with both run and eval subcommands.
**Impact**
The use of --env flag with both run/eval would be more user-friendly as
it gives a precise description of what is not right when using
incorrectly.
If you could give it a look, @dsherret , I appreciate your feedback on
these changes.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Node sets the default HTTP response status code to 200 on the
`ServerResponse`. We initialised it as `undefined` before which caused a
problem with 11ty's dev server.
Thanks to @vrugtehagel for reporting this issue and finding the correct
fix as well 🎉
Fixes https://github.com/denoland/deno/issues/23970
This commit adds initial support for ".npmrc" files.
Currently we only discover ".npmrc" files next to "package.json" files
and discovering these files in user home dir is left for a follow up.
This pass supports "_authToken" and "_auth" configuration
for providing authentication.
LSP support has been left for a follow up PR.
Towards https://github.com/denoland/deno/issues/16105
Fixes #23571.
Previously, we required a `deno.json` to be present (or the `--lock`
flag) in order for us to resolve a `deno.lock` file. This meant that if
you were using deno in an npm-first project deno wouldn't use a
lockfile.
Additionally, while I was fixing that, I discovered there were a couple
bugs keeping the future `install` command from using a lockfile.
With this PR, `install` will actually resolve the lockfile (or create
one if not present), and update it if it's not up-to-date. This also
speeds up `deno install`, as we can use the lockfile to skip work during
npm resolution.
This PR removes the use of the custom `utc_now` function in favor of the
`chrono` implementation. It resolves #22864.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
If `@@asyncIterator` is `null` or `undefined`, it should ignores and
fallback to `@@iterator`.
Tests have been merged into WPT.
https://github.com/web-platform-tests/wpt/pull/46374
The proposal of `ReadableStream.from` uses TC39 [GetIterator][] and
[GetMethod][] within it.
GetMethod treats null as undefined.
So if `@@asyncIterator` is `null` it should be ignored and fallback to
`@@iterator`.
[GetIterator]: https://tc39.es/ecma262/#sec-getiterator
[GetMethod]: https://tc39.es/ecma262/#sec-getmethod
```bash
> deno eval "ReadableStream.from({ [Symbol.asyncIterator]: null, [Symbol.iterator]: () => ({ next: () => ({ done: true }) }) }).pipeTo(new WritableStream())"
error: Uncaught (in promise) TypeError: obj[SymbolAsyncIterator] is not a function
ReadableStream.from({ [Symbol.asyncIterator]: null, [Symbol.iterator]: () => ({ next: () => ({ done: true }) }) }).pipeTo(new WritableStream())
^
at getIterator (ext:deno_web/06_streams.js:5105:38)
at Function.from (ext:deno_web/06_streams.js:5207:22)
at file:///D:/work/js/deno/tests/wpt/suite/$deno$eval:1:16
```
---------
Co-authored-by: Asher Gomez <ashersaupingomez@gmail.com>
This is a primordialization effort to improve resistance against users
tampering with the global `Object` prototype.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Currently `task::task_npx_non_existent` is consistently failing in CI
([example](https://github.com/denoland/deno/actions/runs/9192958846/job/25282900321#step:43:2772))
due to the output changing slightly
```
-- OUTPUT START --
Task non-existent npx this-command-should-not-exist-for-you
npm ERR! code E404
npm ERR! 404 Not Found - GET http://localhost:4260/this-command-should-not-exist-for-you
npm ERR! 404
npm ERR! 404 'this-command-should-not-exist-for-you@*' is not in this registry.
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.
npm ERR! A complete log of this run can be found in: /Users/runner/.npm/_logs/2024-05-22T17_58_42_473Z-debug-0.log
-- OUTPUT END --
-- EXPECTED START --
Task non-existent npx this-command-should-not-exist-for-you
npm error code E404
npm error 404 Not Found - GET http://localhost:4260/this-command-should-not-exist-for-you
[WILDCARD]
-- EXPECTED END --
```
I'm not sure what changed in CI to cause this (and I can't repro it
locally, even matching the version of npm and node on the github
runners), but fix it with more lenient expected output for that test.
By default, uses a 60 second timeout, backing off 2x each time (can be
overridden using the hidden `DENO_SLOW_TEST_TIMEOUT` which we implement
only really for spec testing.
```
Deno.test(async function test() {
await new Promise(r => setTimeout(r, 130_000));
});
```
```
$ target/debug/deno test /tmp/test_slow.ts
Check file:///tmp/test_slow.ts
running 1 test from ../../../../../../tmp/test_slow.ts
test ...'test' is running very slowly (1m0s)
'test' is running very slowly (2m0s)
ok (2m10s)
ok | 1 passed | 0 failed (2m10s)
```
---------
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This brings in [`runtimelib`](https://github.com/runtimed/runtimed) to
use:
## Fully typed structs for Jupyter Messages
```rust
let msg = connection.read().await?;
self
.send_iopub(
runtimelib::Status::busy().as_child_of(msg),
)
.await?;
```
## Jupyter paths
Jupyter paths are implemented in Rust, allowing the Deno kernel to be
installed completely via Deno without a requirement on Python or
Jupyter. Deno users will be able to install and use the kernel with just
VS Code or other editors that support Jupyter.
```rust
pub fn status() -> Result<(), AnyError> {
let user_data_dir = user_data_dir()?;
let kernel_spec_dir_path = user_data_dir.join("kernels").join("deno");
let kernel_spec_path = kernel_spec_dir_path.join("kernel.json");
if kernel_spec_path.exists() {
log::info!("✅ Deno kernel already installed");
Ok(())
} else {
log::warn!("ℹ️ Deno kernel is not yet installed, run `deno jupyter --install` to set it up");
Ok(())
}
}
```
Closes https://github.com/denoland/deno/issues/21619
This stubs `findSourceMap` in `node:module` by always returning
`undefined` as if it never found a source map. This unblocks the `ava`
test runner.
Fixes https://github.com/denoland/deno/issues/18666
Some npm libraries like `signal-exit` rely on the length of the listener
array returned by `process.listeners("SIGNT")` to be correct to
function. We weren't tracking `SIG*` events there, which broke those npm
libraries.
Fixes https://github.com/denoland/deno/issues/22892
the last commit had a regression, where it removed this branch, I
haven't tested the code but I think it should work
---------
Signed-off-by: Bedis Nbiba <bedisnbiba@gmail.com>
Popular test runners like Jest instantiate a new `Process` object
themselves and expect the class constructor to be callable without the
`new` keyword. This PR refactors our `Process` class implementation from
a proper ES2015 class to an ES5-style class which can be invoked both
with and without the `new` keyword like in Node.
Fixes https://github.com/denoland/deno/issues/23863
Construct a new module graph container for workers instead of sharing it
with the main worker.
Fixes #17248
Fixes #23461
---------
Co-authored-by: David Sherret <dsherret@gmail.com>
The stderr stream from the LSP is consumed by a separate thread, so it
may not have processed the part we care about yet. Instead, wait until
you see the measure for the request you care about.
<!--
Before submitting a PR, please read
https://docs.deno.com/runtime/manual/references/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Fixes https://github.com/denoland/deno/issues/23801
---------
Signed-off-by: Marvin Hagemeister <marvinhagemeister50@gmail.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
VScode will typically send a `textDocument/semanticTokens/full` request
followed by `textDocument/semanticTokens/range`, and occassionally
request semantic tokens even when we know nothing has changed. Semantic
tokens also get refreshed on each change. Computing semantic tokens is
relatively heavy in TSC, so we should avoid it as much as possible.
Caches the semantic tokens for open documents, to avoid making TSC do
unnecessary work. Results in a noticeable improvement in local
benchmarking
before:
```
Starting Deno benchmark
-> Start benchmarking lsp
- Simple Startup/Shutdown
(10 runs, mean: 383ms)
- Big Document/Several Edits
(5 runs, mean: 1079ms)
- Find/Replace
(10 runs, mean: 59ms)
- Code Lens
(10 runs, mean: 440ms)
- deco-cx/apps Multiple Edits + Navigation
(5 runs, mean: 9921ms)
<- End benchmarking lsp
```
after:
```
Starting Deno benchmark
-> Start benchmarking lsp
- Simple Startup/Shutdown
(10 runs, mean: 395ms)
- Big Document/Several Edits
(5 runs, mean: 1024ms)
- Find/Replace
(10 runs, mean: 56ms)
- Code Lens
(10 runs, mean: 438ms)
- deco-cx/apps Multiple Edits + Navigation
(5 runs, mean: 8927ms)
<- End benchmarking lsp
```
<!--
Before submitting a PR, please read
https://docs.deno.com/runtime/manual/references/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Fixes https://github.com/denoland/deno/issues/23397
This PR directly addresses the issue raised in #23282 where Deno panics
if `deno coverage` is called with `--include` regex that returns no
matches.
I've opted not to change the return value of `collect_summary` for
simplicity and return an empty `HashMap` instead
Precursor to #23236
This implements the SNI features, but uses private symbols to avoid
exposing the functionality at this time. Note that to properly test this
feature, we need to add a way for `connectTls` to specify a hostname.
This is something that should be pushed into that API at a later time as
well.
```ts
Deno.test(
{ permissions: { net: true, read: true } },
async function listenResolver() {
let sniRequests = [];
const listener = Deno.listenTls({
hostname: "localhost",
port: 0,
[resolverSymbol]: (sni: string) => {
sniRequests.push(sni);
return {
cert,
key,
};
},
});
{
const conn = await Deno.connectTls({
hostname: "localhost",
[serverNameSymbol]: "server-1",
port: listener.addr.port,
});
const [_handshake, serverConn] = await Promise.all([
conn.handshake(),
listener.accept(),
]);
conn.close();
serverConn.close();
}
{
const conn = await Deno.connectTls({
hostname: "localhost",
[serverNameSymbol]: "server-2",
port: listener.addr.port,
});
const [_handshake, serverConn] = await Promise.all([
conn.handshake(),
listener.accept(),
]);
conn.close();
serverConn.close();
}
assertEquals(sniRequests, ["server-1", "server-2"]);
listener.close();
},
);
```
---------
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Moves sloppy import resolution from the loader to the resolver.
Also adds some test helper functions to make the lsp tests less verbose
---------
Co-authored-by: David Sherret <dsherret@gmail.com>
1. Generally we should prefer to use the `log` crate.
2. I very often accidentally commit `eprintln`s.
When we should use `println` or `eprintln`, it's not too bad to be a bit
more verbose and ignore the lint rule.
This PR implements the changes we plan to make to `deno install` in deno
2.0.
- `deno install` without arguments caches dependencies from
`package.json` / `deno.json` and sets up the `node_modules` folder
- `deno install <pkg>` adds the package to the config file (either
`package.json` or `deno.json`), i.e. it aliases `deno add`
- `deno add` can also add deps to `package.json` (this is gated behind
`DENO_FUTURE` due to uncertainty around handling projects with both
`deno.json` and `package.json`)
- `deno install -g <bin>` installs a package as a globally available
binary (the same as `deno install <bin>` in 1.0)
---------
Co-authored-by: Nathan Whitaker <nathan@deno.com>
Fixes the `Debug Failure` errors described in
https://github.com/denoland/deno/issues/23643#issuecomment-2094552765 .
The issue here was that we were passing diagnostic codes as strings but
TSC expects the codes to be numbers. This resulted in some quick fixes
not working (as illustrated by the test added here which fails before
this PR).
The first commit is the actual fix. The rest are just test related.
A bunch of small things, mostly around timing and making sure the
jupyter kernel is actually running and ready to respond to requests. I
reproduced the flakiness by running a script to run a bunch of instances
of the test in parallel, where I could get failures consistently. After
this PR, I can't reproduce the flakiness locally which hopefully means
that applies to CI as well
Allows writing named sub-tests. These are:
1. Filterable on the command line via `cargo test ...`
2. Run in parallel
3. Use a fresh temp and deno dir for each test (unlike steps)
This commit updates our testing npm registry to handle
additional `@denotest2` scope in addition to `@denotest`
scope. I might have to update it further in the future to handle
additional scopes, but it's good enough for now.
Part of #22607 (probably closes it, but I haven't done thorough testing)
Makes it so that `require.resolve` with `paths` specified will fallback
to using the global cache when the paths can't be found when using a
global cache (not when using a node_modules folder)
This correctly creates the `AbortSignal` regardless of when we request
it. If the signal is requested after the request has completed, the
signal is created in the aborted state.
Using GC counts, we can see a reduction in object creation:
This PR: 440
deno 1.42.4: 1650
deno 1.43.0+b02ffec: 874
This PR adds private `[REF]()` and `[UNREF]()` methods to Stdin class,
and call them from Node.js polyfill layer (`TTY` class). This enables
`process.stdin.unref()` and `process.stdin.ref()` for the case when
stdin is terminal.
closes #21796
By default, `deno serve` will assign port 8000 (like `Deno.serve`).
Users may choose a different port using `--port`.
`deno serve /tmp/file.ts`
`server.ts`:
```ts
export default {
fetch(req) {
return new Response("hello world!\n");
},
};
```
When the response has been successfully send, we abort the
`Request.signal` property to indicate that all resources associated with
this transaction may be torn down.
Most common argument to `env` option for `worker_threads.Worker` will be
`process.env`.
In Deno `process.env` is a `Proxy` which can't be cloned using
structured clone algorithm.
So to be safe, I'm creating a copy of actual object before it's sent to
the worker thread.
Ref #23522
This commit adds a "private npm registry" to the test server. This
registry requires to send an appropriate Authorization header.
Towards https://github.com/denoland/deno/issues/16105
This commit changes the workspace support to provide all workspace
members to be available as imports based on their names and versions.
Closes https://github.com/denoland/deno/issues/23343
<!--
Before submitting a PR, please read
https://docs.deno.com/runtime/manual/references/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
This PR wires up a new `jsxPrecompileSkipElements` option in
`compilerOptions` that can be used to exempt a list of elements from
being precompiled with the `precompile` JSX transform.
The actual handling of `$projectChanged` is quick, but JS requests are
not. The cleared caches only get repopulated on the next actual request,
so just batch the change notification in with the next actual request.
No significant difference in benchmarks on my machine, but this speeds
up `did_change` handling and reduces our total number of JS requests (in
addition to coalescing multiple JS change notifs into one).
Embedders may have special requirements around file opening, so we add a
new `check_open` permission check that is called as part of the file
open process.
Adds an `addr` field to `HttpServer` to simplify the pattern
`Deno.serve({ onListen({ port } => listenPort = port })`. This becomes:
`const server = Deno.serve({}); port = server.addr.port`.
Changes:
- Refactors `serve` overloads to split TLS out (in preparation for
landing a place for the TLS SNI information)
- Adds an `addr` field to `HttpServer` that matches the `addr` field of
the corresponding `Deno.Listener`s.
It's not clear to me how these tests worked correctly on CI,
but they were failing hard locally because of two problems:
- missing env var that tests URL for fake npm registry
- trying to run a directory that contains native Node.js tests that
require a special harness
Landing work from #21903, plus fixing a node compat bug.
We were always sending the HTTP/2 ALPN on TLS connections which might
confuse upstream servers.
Changes:
- Configure HTTP/2 ALPN when making the TLS connection from the HTTP/2
code
- Read the `ALPNProtocols` property from the TLS connection options
rather than the deno `alpnProtocols` field
- Add tests
Prereq for landing Deno.serveHttp on Deno.serve: removing older HTTP
servers from the codebase.
This PR enables V8 code cache for ES modules and for `require` scripts
through `op_eval_context`. Code cache artifacts are transparently stored
and fetched using sqlite db and are passed to V8. `--no-code-cache` can
be used to disable.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
It's best that this only gets merged with the latest version of the
suite, so there's little difference between the `ci` and `wpt_epoch`
workflows. This should make troubleshooting easier.
This allows people to use imports like:
```ts
import "./app.css";
```
...with `deno check` in systems where there's a bundle step (ex. Vite).
This will still error when using it with `deno run` or if the referenced
file does not exist.
See test cases for behaviour.
This PR adds a benchmark intended to measure how the LSP handles larger
repos, as well as its performance on a more realistic workload.
The repo being benchmarked is
[deco-cx/apps](https://github.com/deco-cx/apps) which has been vendored
along with its dependencies. It's included as a git submodule as its
fairly large. The LSP requests used in the benchmark are the actual
requests sent by VSCode as I opened, modified, and navigated around a
file (to simulate an actual user interaction).
The main motivation is to have a more realistic benchmark that measures
how we do with a large number of files and dependencies. The
improvements made from 1.42 to 1.42.3 mostly improved performance with
larger repos, so none of our existing benchmarks showed an improvement.
Here are the results for the changes made from 1.42 to 1.42.3 (the new
benchmark is the last one listed):
**1.42.0**
```test
Starting Deno benchmark
-> Start benchmarking lsp
- Simple Startup/Shutdown
(10 runs, mean: 379ms)
- Big Document/Several Edits
(5 runs, mean: 1142ms)
- Find/Replace
(10 runs, mean: 51ms)
- Code Lens
(10 runs, mean: 443ms)
- deco-cx/apps Multiple Edits + Navigation
(5 runs, mean: 25121ms)
<- End benchmarking lsp
```
**1.42.3**
```text
Starting Deno benchmark
-> Start benchmarking lsp
- Simple Startup/Shutdown
(10 runs, mean: 383ms)
- Big Document/Several Edits
(5 runs, mean: 1135ms)
- Find/Replace
(10 runs, mean: 55ms)
- Code Lens
(10 runs, mean: 440ms)
- deco-cx/apps Multiple Edits + Navigation
(5 runs, mean: 11675ms)
<- End benchmarking lsp
```
Closes https://github.com/denoland/deno/issues/23362
Previously we were panicking if there was a pending read on a
port and `receiveMessageOnPort` was called. This is now fixed
by cancelling the pending read, trying to read a message and
resuming reading in a loop.
…faces (#23296)"
This reverts commit e190acbfa8.
Reverting because it broke stable API type declarations. We will reland
it for v1.43 with updated interfaces
Currently we evict a lot of the caches on the JS side of things on every
request, namely script versions, script file names, and compiler
settings (as of #23283, it's not quite every request but it's still
unnecessarily often).
This PR reports changes to the JS side, so that it can evict exactly the
caches that it needs too. We might want to do some batching in the
future so as not to do 1 request per change.
Removes the certificate options from all the interfaces and replaces
them with a new `TlsCertifiedKeyOptions`. This allows us to centralize
the documentation for TLS key management for both client and server, and
will allow us to add key object support in the future.
Also adds an option `keyFormat` field to the cert/key that must be
omitted or set to `pem`. This will allow us to load other format keys in
the future `der`, `pfx`, etc.
In a future PR, we will add a way to load a certified key object, and we
will add another option to `TlsCertifiedKeyOptions` like so:
```ts
export interface TlsCertifiedKeyOptions =
| TlsCertifiedKeyPem
| TlsCertifiedKeyFromFile
| TlsCertifiedKeyConnectTls
| { key: Deno.CertifiedKey }
```
Changes:
- Implements a TCP socket listener that will allow for round-robin
load-balancing in-process.
- Cleans up the raw networking code to make it easier to work with.
