Improving the breadth of collected data, and ensuring that the collected
data is more likely to be successfully reported.
- Use `log` crate in more places
- Hook up `log` crate to otel
- Switch to process-wide otel processors
- Handle places that use `process::exit`
Also adds a more robust testing framework, with a deterministic tracing
setting.
Refs: https://github.com/denoland/deno/issues/26852
This commit makes comments in code snippets in JSDoc or markdown
preserved when they are executed as tests. In particular, this is needed
to get TypeScript special comments such as `@ts-ignore` or
`@ts-expect-error` to work correctly.
Fixes #26728
This will respect `"type": "commonjs"` in a package.json to determine if
`.js`/`.jsx`/`.ts`/.tsx` files are CJS or ESM. If the file is found to
be ESM it will be loaded as ESM though.
Initial import of OTEL code supporting tracing. Metrics soon to come.
Implements APIs for https://jsr.io/@deno/otel so that code using
OpenTelemetry.js just works tm.
There is still a lot of work to do with configuration and adding
built-in tracing to core APIs, which will come in followup PRs.
---------
Co-authored-by: Luca Casonato <hello@lcas.dev>
This PR fixes #24453, by introducing a ctime (using ctime for UNIX and
ChangeTime for Windows) to Deno.stats.
Co-authored-by: Yoshiya Hinosawa <stibium121@gmail.com>
Fixes #26677
Some packages (like supabase) declare bin entries that don't exist until
lifecycle scripts are run. For instance, the lifecycle script downloads
a binary file which serves as a bin entrypoint.
Unfortunately you can't just defer setting up the bin entries until
after lifecycle scripts have run, because the scripts may rely on them.
I looked into this, and PNPM just re-links bin entries after running
lifecycle scripts. I think that's about the best we can do as well.
Note that we'll only re-setup bin entries for packages whose lifecycle
scripts we run. This should limit the performance cost, as typically a
given project will not have many lifecycle scripts (and of those, many
of them probably don't have bin entries to set up).
Fixes https://github.com/denoland/deno/issues/24749
Runs a server that just returns the header tarball and checksum, and
sets the `NODEJS_ORG_MIRROR` env var so that `node-gyp` uses it instead
of `nodejs.org`
This commit adds support for wildcard packages in `workspace`
configuration option in `deno.json`. This is now supported:
```
{
"workspace": [
"./packages/*"
]
}
```
Closes https://github.com/denoland/deno/issues/25783
* cts support
* better cjs/cts type checking
* deno compile cjs/cts support
* More efficient detect cjs (going towards stabilization)
* Determination of whether .js, .ts, .jsx, or .tsx is cjs or esm is only
done after loading
* Support `import x = require(...);`
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Fixes #25342.
Still not sure on the exact user agent to set (should it include
`node`?).
After this PR, here's the state of running some `create-*` packages
(just ones I could think of off the top of my head):
| package | prints/runs/suggests deno install | notes |
| ---------------- | ------------- | ------ |
| `create-next-app` | ❌ | falls back to npm, needs a PR
([code](c32e280209/packages/create-next-app/helpers/get-pkg-manager.ts (L3)))
| `sv create` | ❌ | uses `package-manager-detector`, needs a PR
([code](https://github.com/antfu-collective/package-manager-detector/tree/main))
| `create-qwik` | ✅ | runs `deno install` but suggests `deno start`
which doesn't work (should be `deno task start` or `deno run start`)
| `create-astro` | ✅ | runs `deno install` but suggests `npm run dev`
later in output, probably needs a PR
| `nuxi init` | ❌ | deno not an option in dialog, needs a PR
([code](f04e2e8944/src/commands/init.ts (L96-L102)))
| `create-react-app` | ❌ | uses npm
| `ng new` (`@angular/cli`) | ❌ | uses npm
| `create-vite` | ✅ | suggests working deno commands 🎉
| `create-solid` | ❌ | suggests npm commands, needs PR
It's possible that fixing `package-manager-detector` or other packages
might make some of these just work, but haven't looked too carefully at
each
Fixes https://github.com/denoland/deno/issues/26505
I'm not exactly sure how this case comes about (I tried to write tests
for it but couldn't manage to reproduce it), but what happens is the
parent filename ends up null, and we bail out of resolving the specifier
in package exports.
I've checked, and in node the parent filename is also null (so that's
not a bug on our part), but node continues to resolve even in that case.
So this PR should match node's behavior more closely than we currently
do.
Spend some time stepping through the npm client code and noticed that
the bearer token was different from ours. They do some double encoding
and @dsherret helped me in matching the encoding behavior.
Fixes https://github.com/denoland/deno/issues/26033
Fixes https://github.com/denoland/deno/issues/26509.
Ended up being a `deno_graph` bug causing the error to surface. This PR
updates `deno_graph` to pick up the fix and reverts the temporary
workaround that skipped JSON exports.
While testing, I found out that light-my-request relies on
`ServerResponse.connection`, which is deprecated, so I added that and
`socket`, the non deprecated property.
It also relies on an undocumented `_header` property, apparently for
[raw header
processing](https://github.com/fastify/light-my-request/blob/v6.1.0/lib/response.js#L180-L186).
I added it as an empty string, feel free to provide other approaches.
Fixes #19901
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Fixes playwright on linux, as reported in
https://github.com/denoland/deno/issues/16899#issuecomment-2378268454.
The issue was that we were opening the socket in nonblocking mode, which
meant that subprocesses trying to use it would get a `EWOULDBLOCK` error
(unexpectedly). The fix here is to only set nonblocking mode on our end
(which we need to use asynchronously)
Fixes #26498.
This was a sort of intentional decision originally, as I wanted to avoid
caching extra files that may not be needed. It seems like that behavior
is unintuitive, so I propose we cache all of the exports of listed jsr
packages when you run a bare `deno install`.
This commit makes sure that `deno add`, `deno install` and `deno remove`
update the lockfile if only `package.json` file is present.
Fixes https://github.com/denoland/deno/issues/26270
1. Respects the formatting of the file (ex. keeps four space indents or
tabs).
2. Handles editing of comments.
3. Handles trailing commas.
4. Code is easier to maintain.
This PR fixes the issue where mapped specifiers in a workspace member
would never be found. Only mapped paths from the workspace root would
resolve.
This was caused by always passing the workspace root url to the import
map resolver instead of the workspace member one.
Fixes https://github.com/denoland/deno/issues/26138
Fixes https://github.com/denoland/fresh/issues/2615
---------
Signed-off-by: Marvin Hagemeister <marvinhagemeister50@gmail.com>
Co-authored-by: David Sherret <dsherret@users.noreply.github.com>
<!--
Before submitting a PR, please read
https://docs.deno.com/runtime/manual/references/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Fixes https://github.com/denoland/deno/issues/26115.
We weren't normalizing the headers to lower case, so code that attempted
to delete the `Content-Length` header (but used a different case) wasn't
actually removing the header.
partially unblocks #25470
This PR aligns the resolution of `localhost` hostname to Node.js
behavior.
In Node.js `dns.lookup("localhost", (_, addr) => console.log(addr))`
prints ipv6 address `::1`, but it prints ipv4 address `127.0.0.1` in
Deno. That difference causes some errors in the work of enabling
`createConnection` option in `http.request` (#25470). This PR fixes the
issue by aligning `dns.lookup` behavior to Node.js.
This PR also changes the following behaviors (resolving TODOs):
- `http.createServer` now listens on ipv6 address `[::]` by default on
linux/mac
- `net.createServer` now listens on ipv6 address `[::]` by default on
linux/mac
These changes are also alignments to Node.js behaviors.
`napi_call_function` should use our equiv of NAPI_PREAMBLE (`&mut Env`
instead of `*mut Env`) since it needs to set error codes based on
whether the body of the function raised a JS exception.
Fixes: https://github.com/denoland/deno/issues/26282
This came up on Discord as a question so I thought it's worth adding a
hint for this as it might be a common pitfall.
---------
Signed-off-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: David Sherret <dsherret@users.noreply.github.com>
Fixes https://github.com/denoland/deno/issues/26119.
Originally I wanted to put them in package.json if there's no deno.json,
but on second thought it makes more sense to just create a deno.json
Fixes https://github.com/denoland/deno/issues/26177
The significant delay was caused by Nagel's algorithm + delayed ACKs in
Linux kernels. Here's the [kernel
patch](https://lwn.net/Articles/502585/) which added 40ms
`tcp_default_delack_min`
```
$ deno run -A pg-bench.mjs # main
Tue Oct 15 2024 12:27:22 GMT+0530 (India Standard Time): 42ms
$ target/release/deno run -A pg-bench.mjs # this patch
Tue Oct 15 2024 12:28:02 GMT+0530 (India Standard Time): 1ms
```
```js
import { Buffer } from "node:buffer";
import pg from 'pg'
const { Client } = pg
const client = new Client({
connectionString: 'postgresql://postgres:postgres@127.0.0.1:5432/postgres'
})
await client.connect()
async function fetch() {
const startPerf = performance.now();
const res = await client.query(`select
$1::int as int,
$2 as string,
$3::timestamp with time zone as timestamp,
$4 as null,
$5::bool as boolean,
$6::bytea as bytea,
$7::jsonb as json
`, [
1337,
'wat',
new Date().toISOString(),
null,
false,
Buffer.from('awesome'),
JSON.stringify([{ some: 'json' }, { array: 'object' }])
])
console.log(`${new Date()}: ${Math.round(performance.now() - startPerf)}ms`)
}
for(;;) await fetch();
```
When using the `--unstable-detect-cjs` flag or adding `"unstable":
["detect-cjs"]` to a deno.json, it will make a JS file CJS if the
closest package.json contains `"type": "commonjs"` and the file is not
an ESM module (no TLA, no `import.meta`, no `import`/`export`).
Fixes #22995. Fixes #23000.
There were a handful of bugs here causing the hang (each with a
corresponding minimized test):
- We were canceling recv futures when `receiveMessageOnPort` was called,
but this caused the "receive loop" in the message port to exit. This was
due to the fact that `CancelHandle`s are never reset (i.e., once you
`cancel` a `CancelHandle`, it remains cancelled). That meant that after
`receieveMessageOnPort` was called, the subsequent calls to
`op_message_port_recv_message` would throw `Interrupted` exceptions, and
we would exit the loop.
The cancellation, however, isn't actually necessary.
`op_message_port_recv_message` only borrows the underlying port for long
enough to poll the receiver, so the borrow there could never overlap
with `op_message_port_recv_message_sync`.
- Calling `MessagePort.unref()` caused the "receive loop" in the message
port to exit. This was because we were setting
`messageEventListenerCount` to 0 on unref. Not only does that break the
counter when multiple `MessagePort`s are present in the same thread, but
we also exited the "receive loop" whenever the listener count was 0. I
assume this was to prevent the recv promise from keeping the event loop
open.
Instead of this, I chose to just unref the recv promise as needed to
control the event loop.
- The last bug causing the hang (which was a doozy to debug) ended up
being an unfortunate interaction between how we implement our
messageport "receive loop" and a pattern found in `npm:piscina` (which
angular uses). The gist of it is that piscina uses an atomic wait loop
along with `receiveMessageOnPort` in its worker threads, and as the
worker is getting started, the following incredibly convoluted series of
events occurs:
1. Parent sends a MessagePort `p` to worker
2. Parent sends a message `m` to the port `p`
3. Parent notifies the worker with `Atomics.notify` that a new message
is available
4. Worker receives message, adds "message" listener to port `p`
5. Adding the listener triggers `MessagePort.start()` on `p`
6. Receive loop in MessagePort.start receives the message `m`, but then
hits an await point and yields (before dispatching the "message" event)
7. Worker continues execution, starts the atomic wait loop, and
immediately receives the existing notification from the parent that a
message is available
8. Worker attempts to receive the new message `m` with
`receiveMessageOnPort`, but this returns `undefined` because the receive
loop already took the message in 6
9. Atomic wait loop continues to next iteration, waiting for the next
message with `Atomic.wait`
10. `Atomic.wait` blocks the worker thread, which prevents the receive
loop from continuing and dispatching the "message" event for the
received message
11. The parent waits for the worker to respond to the first message, and
waits
12. The thread can't make any more progress, and the whole process hangs
The fix I've chosen here (which I don't particularly love, but it works)
is to just delay the `MessagePort.start` call until the end of the event
loop turn, so that the atomic wait loop receives the message first. This
prevents the hang.
---
Those were the main issues causing the hang. There ended up being a few
other small bugs as well, namely `exit` being emitted multiple times,
and not patching up the message port when it's received by
`receiveMessageOnPort`.
Although using `--allow-run` without an allow list gives basically no
security, I think we should remove this warning because it gets in the
way and the only way to disable it is via --quiet.
Closes https://github.com/denoland/deno/issues/26012
```
========================================
failures:
"/WebCryptoAPI/wrapKey_unwrapKey/wrapKey_unwrapKey.https.any.html - Can wrap and unwrap X25519 private key keys as non-extractable using pkcs8 and AES-CTR"
final result: failed. 1 passed; 1 failed; 0 expected failure; total 2 (15646ms)
diff --git a/Users/divy/gh/deno/tests/wpt/runner/expectation.json b/var/folders/ll/ltqsx4nx72v5_r7rlsl36pgm0000gn/T/375d79f1257b05cd
index fb2063935..4449c5d15 100644
--- a/Users/divy/gh/deno/tests/wpt/runner/expectation.json
+++ b/var/folders/ll/ltqsx4nx72v5_r7rlsl36pgm0000gn/T/375d79f1257b05cd
@@ -1531,6 +1531,7 @@
},
"wrapKey_unwrapKey": {
"wrapKey_unwrapKey.https.any.html": [
+ "Can wrap and unwrap X25519 private key keys as non-extractable using pkcs8 and AES-CTR",
"Can wrap and unwrap X25519 private key keys as non-extractable using pkcs8 and AES-CBC",
"Can wrap and unwrap X25519 private key keys as non-extractable using pkcs8 and AES-GCM",
"Can wrap and unwrap X25519 private key keys as non-extractable using pkcs8 and AES-KW",
```
Fixes #25998. Fixes https://github.com/denoland/deno/issues/25928.
Originally I was just going to make this an error message instead of a
panic, but once I got to a minimal repro I felt that this really should
work.
The panic occurs when you have `nodeModulesDir: manual` (or a
package.json present), and you have an npm package with a tag in your
deno.json (see the spec test that illustrates this).
This code path only actually executes when trying to choose an
appropriate package version from `node_modules/.deno`, so we should be
able to fix it by storing some extra data at install time.
The fix proposed here is to repurpose the `.initialized` file that we
store in `node_modules` to store the tags associated with a package.
Basically, if you have a version requirement with a tag (e.g.
`npm:chalk@latest`), when we set up the node_modules folder for that
package, we store the tag (`latest`) in `.initialized`. Then, when doing
BYONM resolution, if we have a version requirement with a tag, we read
that file and check if the tag is present.
The downside is that we do more work when setting up `node_modules`. We
_could_ do this only when BYONM is enabled, but that would have the
downside of needing to re-run `deno install` when you switch from auto
-> manual, though maybe that's not a big deal.
Fixes #25861.
Previously we were attempting to match the version requirement against
the version already present in `node_modules` root, and if they didn't
match we would create a node_modules dir in the workspace member's
directory with the dependency.
Aside from the fact that this caused the panic, on second thought it
just doesn't make sense in general. We shouldn't be semver matching, as
resolution has already occurred and decided what package versions are
required. Instead, we can just compare the versions directly.
Fixes #24740.
Implements the `uv_mutex_*` and `uv_async_*` APIs.
The mutex API is implemented exactly as libuv, a thin wrapper over the
OS's native mutex.
The async API is implemented in terms of napi_async_work. As documented
in the napi docs, you really shouldn't call `napi_queue_async_work`
multiple times (it is documented as undefined behavior). However, our
implementation doesn't have any issue with this, so I believe it suits
our purpose here.
Previously the CLI was incorrectly reporting `React` as unused in a JSX
file that uses the "old" transform.
The LSP was already handling this correctly.
Currently we only warn once. With this PR, we continue to warn about
not-run scripts on explicit `deno install` (or cache). For `run` (or
other subcommands) we only warn the once, as we do currently.
Fixes https://github.com/denoland/deno/issues/25862.
npm only makes bin entries executable if they get linked into `.bin`, as
we did before this PR. So this PR actually deviates from npm, because
it's the only reasonable way to fix this that I can think of.
---
The reason this was broken in moment is the following:
Moment has dependencies on two typescript versions: 1.8 and 3.1
If you have two packages with conflicting bin entries (i.e. two
typescript versions which both have a bin entry `tsc`), in npm it is
non-deterministic and undefined which one will end up in `.bin`.
npm, due to implementation differences, chooses to put typescript 1.8
into the `.bin` directory, and so `node_modules/typescript/bin/tsc` ends
up getting marked executable. We, however, choose typescript 3.2, and so
we end up making `node_modules/typescript3/bin/tsc` executable.
As part of its tests, moment executes `node_modules/typescript/bin/tsc`.
Because we didn't make it executable, this fails.
Since the conflict resolution is undefined in npm, instead of trying to
match it, I think it makes more sense to just make bin entries
executable even if they aren't chosen in the case of a conflict.
