1
0
Fork 0
mirror of https://github.com/denoland/deno.git synced 2024-12-18 05:14:21 -05:00
Commit graph

4842 commits

Author SHA1 Message Date
Bartek Iwańczuk
972469d309
tests: disable another flaky test (#18762)
This test is flaky too, it actually shouldn't be running since it's not
listed in `cli/tests/node_compat/config.jsonc` at all.
2023-04-19 00:10:01 +00:00
Bartek Iwańczuk
edca01c35e
chore: disable flaky Node compat tests (#18760)
I'm not able to reproduce any of the failures from CI on my machine.

I'm going to disable these tests for now as they are holding us back.
2023-04-19 00:32:21 +02:00
Bartek Iwańczuk
40d9e918ea
test: fix flaky worker test (#18754)
https://github.com/denoland/deno/actions/runs/4734585198/jobs/8403719901?pr=18753
2023-04-18 22:24:47 +02:00
denobot
4e5d370b56
chore: forward v1.32.5 release commit to main (#18758)
Co-authored-by: levex <levex@users.noreply.github.com>
Co-authored-by: Levente Kurusa <lkurusa@kernelstuff.org>
2023-04-18 22:20:49 +02:00
Levente Kurusa
530963c34c
refactor(node/crypto): scrypt polyfill to rust (#18746) 2023-04-18 14:29:10 +02:00
Yoshiya Hinosawa
74aee8b305
chore(ext/node): restore http_test from std (#18747) 2023-04-18 21:06:25 +09:00
Yoshiya Hinosawa
5f44396a9e
fix(ext/node): implement crypto.createVerify (#18703) 2023-04-18 21:04:51 +09:00
Yoshiya Hinosawa
de21a780fd
fix(ext/node): add req.socket.remoteAddress (#18733) 2023-04-18 16:00:51 +09:00
Yoshiya Hinosawa
6efaef606a
fix(ext/node): polyfill response._implicitHeader method (#18738) 2023-04-18 15:20:36 +09:00
David Sherret
d2d62b6312
refactor(npm): add CliNodeResolver (#18742) 2023-04-17 15:36:23 -04:00
Yoshiya Hinosawa
559a61b879
chore: update node compat config (#18736) 2023-04-18 00:36:49 +09:00
Cre3per
bad4b7554b
refactor(tests): Watcher test timeout (#18459)
Closes #17438
2023-04-17 09:49:03 -04:00
David Sherret
94744ae25a
fix(lsp): ensure language server status works on unix (#18727)
Closes #18724
2023-04-16 16:02:48 -04:00
Bartek Iwańczuk
02dd09a36f
bench: add benchmark for deferred async ops (#18722)
```
./target/release/deno run cli/bench/async_ops_deferred.js
time 794 ms rate 1259445
time 786 ms rate 1272264
time 770 ms rate 1298701
time 784 ms rate 1275510
time 775 ms rate 1290322
time 786 ms rate 1272264
time 773 ms rate 1293661
time 771 ms rate 1297016
time 774 ms rate 1291989
time 767 ms rate 1303780
time 764 ms rate 1308900
time 768 ms rate 1302083
time 763 ms rate 1310615
time 761 ms rate 1314060
time 761 ms rate 1314060
time 762 ms rate 1312335
time 763 ms rate 1310615
time 759 ms rate 1317523
time 760 ms rate 1315789
time 761 ms rate 1314060
time 769 ms rate 1300390
time 763 ms rate 1310615
time 760 ms rate 1315789
time 763 ms rate 1310615
time 761 ms rate 1314060
time 759 ms rate 1317523
time 765 ms rate 1307189
time 760 ms rate 1315789
time 764 ms rate 1308900
time 763 ms rate 1310615
time 760 ms rate 1315789
time 757 ms rate 1321003
time 763 ms rate 1310615
time 759 ms rate 1317523
time 771 ms rate 1297016
time 759 ms rate 1317523
time 759 ms rate 1317523
time 763 ms rate 1310615
time 754 ms rate 1326259
time 755 ms rate 1324503
time 762 ms rate 1312335
time 752 ms rate 1329787
time 755 ms rate 1324503
time 754 ms rate 1326259
time 759 ms rate 1317523
time 754 ms rate 1326259
time 749 ms rate 1335113
time 753 ms rate 1328021
time 756 ms rate 1322751
time 753 ms rate 1328021
```

```
samply record -r 20000 target/release/deno run cli/bench/async_ops_deferred.js
```

https://share.firefox.dev/43Efvm6
2023-04-16 19:47:06 +00:00
Bartek Iwańczuk
1564595f6b
Revert "perf(core): immediately schedule another tick if there are un… (#18718)
…polled ops (#18611)"

This reverts commit 4317055a3e.

Reverting because we discovered while working on
https://github.com/denoland/deno/pull/18619 that it doesn't work
correctly (sometimes waker just stops working).
2023-04-16 14:05:13 +00:00
Ryan Dahl
2184103a5e
feat(kv): AtomicOperation#sum (#18704) 2023-04-15 10:33:31 +02:00
David Sherret
1bca994143
chore: fix running node tests in parallel on Windows (#18711) 2023-04-14 18:50:58 -04:00
David Sherret
0a67a3965f
refactor: add TypeChecker struct (#18709)
Adds a `TypeChecker` struct and pushes more shared functionality into
it.
2023-04-14 18:05:46 -04:00
David Sherret
f6a28e3e62
chore: remove node_modules folder being created in testdata directory (#18708) 2023-04-14 21:56:14 +00:00
David Sherret
136dce67ce
refactor: break up ProcState (#18707)
1. Breaks up functionality within `ProcState` into several other structs
to break out the responsibilities (`ProcState` is only a data struct
now).
2. Moves towards being able to inject dependencies more easily and have
functionality only require what it needs.
3. Exposes `Arc<T>` around the "service structs" instead of it being
embedded within them. The idea behind embedding them was to reduce the
verbosity of needing to pass around `Arc<...>`, but I don't think it was
exactly working and as we move more of these structs to be more
injectable I don't think the extra verbosity will be a big deal.
2023-04-14 16:22:33 -04:00
Divy Srivastava
a411144219
fix(ext/websocket): Avoid write deadlock that requires read_frame to complete (#18705)
Fixes https://github.com/denoland/deno/issues/18700

Timeline of the events that lead to the bug.

1. WebSocket handshake complete
2. Server on `read_frame` holding an AsyncRefCell borrow of the
WebSocket stream.
3. Client sends a TXT frame after a some time
4. Server recieves the frame and goes back to `read_frame`.
5. After some time, Server starts a `write_frame` but `read_frame` is
still holding a borrow!
^--- Locked. read_frame needs to complete so we can resume the write.

This commit changes all writes to directly borrow the
`fastwebsocket::WebSocket` resource under the assumption that it won't
affect ongoing reads.
2023-04-14 22:17:39 +05:30
Bartek Iwańczuk
cb2ca234bb
refactor(core): limit number of boundary crossings between Rust and V8 (#18652)
This commit refactors "deno_core" to do fewer boundary crossings
from Rust to V8. In other words we are now calling V8 from Rust fewer
times.

This is done by merging 3 distinct callbacks into a single one. Instead
of having "op resolve" callback, "next tick" callback and "macrotask
queue" callback, we now have only "Deno.core.eventLoopTick" callback,
which is responsible for doing the same actions previous 3 callbacks.

On each of the event loop we were doing at least 2 boundary crosses
(timers macrotask queue callback and unhandled promise rejection
callback) and up to 4 crosses if there were op response and next tick
callbacks coming from Node.js compatibility layer. Now this is all done
in a single callback.

Closes https://github.com/denoland/deno/issues/18620
2023-04-14 02:41:32 +02:00
David Sherret
d192d84a0e
refactor(cli): add Emitter struct (#18690)
Removes the functions in the `emit` module and replaces them with an
`Emitter` struct that can have "ctor dependencies" injected rather than
using functions to pass along the dependencies.

This is part of a long term refactor to move more functionality out of
proc state.
2023-04-13 18:03:07 +00:00
Nayeem Rahman
6e8618ae0f
refactor(cli): move runTests() and runBenchmarks() to rust (#18563)
Stores the test/bench functions in rust op state during registration.
The functions are wrapped in JS first so that they return a directly
convertible `TestResult`/`BenchResult`. Test steps are still mostly
handled in JS since they are pretty much invoked by the user. Allows
removing a bunch of infrastructure for communicating between JS and
rust. Allows using rust utilities for things like shuffling tests
(`Vec::shuffle`). We can progressively move op and resource sanitization
to rust as well.

Fixes #17122.
Fixes #17312.
2023-04-13 19:43:23 +02:00
David Sherret
4e53bc5a94
chore: bump child_process_test timeouts for slow CI (#18689) 2023-04-13 13:30:30 -04:00
David Sherret
efa7c19890
refactor: upgrade to deno_npm 0.3.0 (#18671)
This allows us to specify the `@types/node` version constraint in the
CLI instead of in deno_npm.
2023-04-13 10:47:45 -04:00
Bartek Iwańczuk
702284dc22
perf(ops): directly respond for eager ops (#18683)
This commit changes "eager ops" to directly return a response value
instead of calling "opresponse" callback in JavaScript. This saves
one boundary crossing and has a fantastic impact on the "async_ops.js"
benchmark:

```
v1.32.4
$ deno run cli/bench/async_ops.js
time 329 ms rate 3039513
time 322 ms rate 3105590
time 307 ms rate 3257328
time 301 ms rate 3322259
time 303 ms rate 3300330
time 306 ms rate 3267973
time 300 ms rate 3333333
time 301 ms rate 3322259
time 301 ms rate 3322259
time 301 ms rate 3322259
time 302 ms rate 3311258
time 301 ms rate 3322259
time 302 ms rate 3311258
time 302 ms rate 3311258
time 303 ms rate 3300330
```

```
this branch
$ ./target/release/deno run -A cli/bench/async_ops.js
time 257 ms rate 3891050
time 248 ms rate 4032258
time 251 ms rate 3984063
time 246 ms rate 4065040
time 238 ms rate 4201680
time 227 ms rate 4405286
time 228 ms rate 4385964
time 229 ms rate 4366812
time 228 ms rate 4385964
time 226 ms rate 4424778
time 226 ms rate 4424778
time 227 ms rate 4405286
time 228 ms rate 4385964
time 227 ms rate 4405286
time 228 ms rate 4385964
time 227 ms rate 4405286
time 229 ms rate 4366812
time 228 ms rate 4385964
```

Prerequisite for https://github.com/denoland/deno/pull/18652
2023-04-13 14:32:47 +02:00
Geert-Jan Zwiers
aaebd6eb42
chore(docs): clarify what subcommands do not type-check by default (#18520)
The CLI docs suggested that all deno subcommands no longer type-check by
default. This is only the case for some subcommands, and this PR
clarifies the CLI docs in this regard.
2023-04-13 03:42:28 +02:00
Yiyu Lin
d790ea7d53
refactor(cli,ext,ops): cleanup regex with lazy-regex (#17296)
- bump deps: the newest `lazy-regex` need newer `oncecell` and
`regex`
- reduce `unwrap`
- remove dep `lazy_static`
- make more regex cached

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-04-13 03:08:01 +02:00
Elijah
e2853a955c
test(runtime): support Windows in resolve_from_cwd_absolute (#18379)
`current_dir().unwrap()` joined with a Path is equivalent to the
implementation in `resolve_from_cwd()`. Manually tested on Ubuntu 22.04
and Windows 11.

Signed-off-by: Elijah Conners <business@elijahpepe.com>
2023-04-13 01:45:53 +02:00
Nayeem Rahman
d97f9d22b3
fix(test): add process sigint handler for --watch (#18678)
Fixes #18676.
2023-04-13 00:51:04 +02:00
David Sherret
17e4782140
fix(npm): eagerly reload package information when version from lockfile not found locally (#18673)
Closes #18624
2023-04-12 18:04:45 -04:00
David Sherret
9c255b2843
refactor: ProcState::build -> ProcState::from_flags (#18672) 2023-04-12 14:54:28 -04:00
denobot
05c393b99b
chore: forward v1.32.4 release commit to main (#18669)
Co-authored-by: levex <levex@users.noreply.github.com>
2023-04-12 13:03:27 -04:00
Luca Casonato
f90caa821c
refactor(ext/fs): abstract FS via FileSystem trait (#18599)
This commit abstracts out the specifics of the underlying system calls
FS operations behind a new `FileSystem` and `File` trait in the
`ext/fs` extension.

This allows other embedders to re-use ext/fs, but substituting in a
different FS backend.

This is likely not the final form of these traits. Eventually they will
be entirely `deno_core::Resource` agnostic, and will live in a seperate
crate.

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-04-12 15:13:32 +02:00
David Sherret
0e3f62d444
fix(npm): cache bust npm specifiers more aggressively (#18636)
Part 1: #18622 
Part 2: This PR

Closes #16901

---------

Co-authored-by: Luca Casonato <hello@lcas.dev>
2023-04-12 08:36:11 -04:00
Nayeem Rahman
806671af33
fix(core): preserve syntax error locations in dynamic imports (#18664)
Fixes #6259.

Adds the location for v8 syntax errors to the message (`message += " at
{location}"`) when rethrowing them for dynamic imports.

Discussing with @bartlomieju on discord I proposed just preserving v8's
error and not reconstructing it, allowing the standard stack trace to
just point to the syntax error instead of the dynamic import. But on
further thought this way has parity with SWC's syntax errors + has the
advantage of showing both the syntax error and dynamic import location.

```ts
// temp.js
await import("./temp2.js");

// temp2.js
function foo() {
  await Promise.resolve();
}

// Before:
// error: Uncaught (in promise) SyntaxError: Unexpected reserved word
// await import("./temp2.js");
// ^
//     at async file:///.../temp.js:1:1

// After:
// error: Uncaught (in promise) SyntaxError: Unexpected reserved word at file:///.../temp2.js:2:3
// await import("./temp2.js");
// ^
//     at async file:///.../temp.js:1:1
```
2023-04-12 13:45:36 +02:00
Satya Rohith
18e0ee368c
fix(ext/cache): cache.put overwrites previous call (#18649) 2023-04-12 11:25:19 +05:30
Levente Kurusa
65b0d2316d
refactor(node/crypto): port polyfill to Rust for randomInt, randomFill, randomFillSync (#18658)
Pretty much as per the title, I'd welcome some feedback especially
around the
array/buffer handling in the two randomFill functions.
2023-04-12 02:57:57 +02:00
David Sherret
8820f6e922
fix(npm): do not "npm install" when npm specifier happens to match package.json entry (#18660) 2023-04-11 18:10:51 -04:00
Geert-Jan Zwiers
805214626f
chore(cli): clean up unused number value (#18661)
This PR removes an accidentally declared number value.
2023-04-11 16:00:05 -04:00
Luca Casonato
e0bf8e6faf
fix(ext/kv): keys must be arrays (#18655)
There was some leftover code from previous iterations, where keys could
be single parts instead of arrays also. This didn't match the types.
2023-04-11 14:36:01 +00:00
Divy Srivastava
a0dd0cbcb9
fix(ext/node): add X509Certificate (#18625)
Towards #18455
2023-04-07 22:54:16 +05:30
David Sherret
5c7f76c570
fix(npm): reload an npm package's dependency's information when version not found (#18622)
This reloads an npm package's dependency's information when a
version/version req/tag is not found.

This PR applies only to dependencies of npm packages. It does NOT yet
cause npm specifiers to have their dependency information cache busted.
That requires a different solution, but this should help cache bust in
more scenarios.

Part of #16901, but doesn't close it yet
2023-04-06 21:41:19 -04:00
David Sherret
d07aa4a072
refactor(npm): use deno_npm and deno_semver (#18602) 2023-04-06 18:46:44 -04:00
David Sherret
1586c52b5b
chore: fix flaky tests checking ms when CI is slow (#18559) 2023-04-06 23:48:07 +02:00
Divy Srivastava
df72420d72
fix(ext/node): implement hkdf-expand (#18612)
Towards https://github.com/denoland/deno/issues/18455
2023-04-06 16:56:56 +00:00
Marvin Hagemeister
e51985ca74
fix(ext/node): fix unable to resolve fraction.js (#18544)
Turns out `autoprefixer` is a better reproduction case then
`microbundle`.

Fixes #18535 
Fixes #18600

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-04-06 16:12:50 +02:00
Divy Srivastava
3b62a58818
fix(ext/node): add symmetric keygen (#18609)
Towards #18455
2023-04-06 18:39:25 +05:30
Yusuke Tanaka
9626c48a01
feat(task): introduce built-in unset command to deno task (#18606)
This introduces a new built-in `unset` command to `deno task` by bumping
deno_task_shell to the latest version 0.11.0. Also this includes a fix
on how `deno task` handles empty environment variables (see
https://github.com/denoland/deno_task_shell/pull/80 for details).
2023-04-06 11:53:44 +09:00
Geert-Jan Zwiers
a7e25b8126
fix(ext/node): json encode binary command name (#18596)
Fixes https://github.com/denoland/deno/issues/18588
2023-04-06 01:46:21 +02:00
Divy Srivastava
34d596e04f
chore(cli/bench): add ws echo bench (#18595) 2023-04-05 18:31:07 +05:30
Bartek Iwańczuk
db39855fcb
tests: cleanup "node_compat_tests" (#18594)
A few drive-by cleanup while I'm working on the "crypto"
module. It makes it easier and faster to debug the failing
test case.
2023-04-05 13:15:57 +02:00
Bartek Iwańczuk
c4628aa809
refactor(ext/http): bring back 'reusePort' option for 'Deno.serve()' (#18590)
Closes https://github.com/denoland/deno/issues/18582
2023-04-04 14:17:36 +00:00
Matt Mastracci
a1764f7690
refactor(core): Improve ergonomics of managing ASCII strings (#18498)
This is a follow-on to the earlier work in reducing string copies,
mainly focused on ensuring that ASCII strings are easy to provide to the
JS runtime.

While we are replacing a 16-byte reference in a number of places with a
24-byte structure (measured via `std::mem::size_of`), the reduction in
copies wins out over the additional size of the arguments passed into
functions.

Benchmarking shows approximately the same if not slightly less wallclock
time/instructions retired, but I believe this continues to open up
further refactoring opportunities.
2023-04-04 06:46:31 -06:00
Kenta Moriuchi
2dc2016837
feat(ext/url): URL.canParse (#18286) 2023-04-04 13:34:12 +02:00
Dj
62c5664697
feat(ext/ffi): support marking symbols as optional (#18529) 2023-04-03 21:32:21 +03:00
Bartek Iwańczuk
51d3fb78ad
refactor: remove "ext/flash" (#18578)
With https://github.com/denoland/deno/pull/18568 landed we no longer
need "ext/flash". 

This commit removes "deno_flash" extension completely.

This should have some impact on the binary and snapshot size.

Closes https://github.com/denoland/deno/issues/17356
2023-04-03 19:01:02 +02:00
Bartek Iwańczuk
2846bbe0a3
refactor: "Deno.serve()" API uses "Deno.serveHttp()" internally (#18568)
This commit changes implementation of "Deno.serve()" API to use
"Deno.serveHttp()" under the hood. This change will allow us to
remove the "flash" server implementation, bringing stability to the
"Deno.serve()" API.

"cli/tests/unit/flash_test.ts" was renamed to "serve_test.ts".

Closes https://github.com/denoland/deno/issues/15574
Closes https://github.com/denoland/deno/issues/15504
Closes https://github.com/denoland/deno/issues/15646
Closes https://github.com/denoland/deno/issues/15909
Closes https://github.com/denoland/deno/issues/15911
Closes https://github.com/denoland/deno/issues/16828
Closes https://github.com/denoland/deno/issues/18046
Closes https://github.com/denoland/deno/issues/15869
2023-04-03 17:44:18 +02:00
Yoshiya Hinosawa
6d68392f8a
Revert "fix(cli): don't store blob and data urls in the module cache (#18261)" (#18572)
This reverts commit b4c61c146a.

cc @nayeemrmn
2023-04-03 21:05:39 +09:00
denobot
ad8d0c90d1
chore: forward v1.32.3 release commit to main (#18561)
Co-authored-by: David Sherret <dsherret@gmail.com>
2023-04-01 20:27:53 -04:00
Nayeem Rahman
e5588d2f1e
fix(test): don't swallow sanitizer errors with permissions (#18550)
Missing `return` from #18246.
2023-04-01 16:20:16 -04:00
David Sherret
0210c1cadf
fix(lsp): add a document preload file system entry limit (#18553)
I was testing this out and it's badly needed. For now, it's not
configurable and limited to 1,000 file system entries.

Related to #18538
2023-04-01 15:10:30 -04:00
David Sherret
bac8e4f6f2
fix(repl): disable language server document preloading in the repl (#18543)
This was an oversight because the repl uses the language server under
the hood. Also, never preloads from a root directory.

Part of #18538
2023-04-01 12:02:44 -04:00
David Sherret
ae1ba2af3c
perf(check): faster source hashing (#18534) 2023-04-01 10:12:40 -04:00
David Sherret
23b9be7b37
fix(check): ensure diagnostics caused by changes in other files get invalidated between runs (#18541)
Regression caused by the performance improvement in #18329. Figuring
this out was hard. It's luckily still fast after this change.

Closes #18516
2023-04-01 10:04:56 -04:00
denobot
f465123fdc
chore: forward v1.32.2 release commit to main (#18539)
This is the release commit being forwarded back to main for 1.32.2
2023-04-01 00:10:13 +00:00
Bartek Iwańczuk
cbd14085e6
Revert "fix(cli): deno upgrade file permission (#18427)" (#18467)
This reverts commit 0742ea1170.

Closes https://github.com/denoland/deno/issues/18466
2023-03-31 13:43:45 -04:00
David Sherret
7ec45770c8
fix: upgrade to TypeScript 5.0.3 (#18532) 2023-03-31 17:09:04 +00:00
David Sherret
87ccd4bcd1
fix(lsp): better handling of data: urls (#18527)
1. Log instead of error when the referrer can't be found
2. Fixes typescript to resolve data urls correctly. Properly documented
here:
https://github.com/denoland/TypeScript/pull/4/files#diff-180da7c288743d11d8590d30f0c07c48e5dcf291aa671bbea0dd520a9a1359d2

Closes #18524
2023-03-31 16:48:18 +00:00
David Sherret
7722014497
fix(lsp): include all diagnosable documents on initialize (#17979)
Closes https://github.com/denoland/vscode_deno/issues/797
Closes https://github.com/denoland/deno/issues/11190
Closes https://github.com/denoland/vscode_deno/issues/811
Closes https://github.com/denoland/vscode_deno/issues/761
Closes https://github.com/denoland/vscode_deno/issues/585
Closes https://github.com/denoland/vscode_deno/issues/561
Closes https://github.com/denoland/vscode_deno/issues/410
2023-03-30 17:47:53 -04:00
Luca Casonato
02e01b171f
fix(dts): improve types for the Deno.KV API (#18510) 2023-03-30 22:52:31 +02:00
Luca Casonato
e888c3f534
feat(ext/kv): return versionstamp from set/commit (#18512)
This commit updates the `Deno.Kv` API to return the new commited
versionstamp for the mutated data from `db.set` and `ao.commit`. This is
returned in the form of a `Deno.KvCommitResult` object that has a
`versionstamp` property.
2023-03-30 20:57:21 +02:00
Geert-Jan Zwiers
206c593519
fix(coverage): ignore files from npm registry (#18457)
Fixes https://github.com/denoland/deno/issues/17664 and part of
https://github.com/denoland/deno/issues/18454 by excluding files
belonging to npm modules by default in the coverage output.
2023-03-30 17:40:22 +00:00
David Sherret
c4f82cab31
fix(lsp): textDocument/references should respect includeDeclaration (#18496) 2023-03-30 16:15:21 +00:00
David Sherret
3abc53f811
docs: clarify Deno.consoleSize returns the window size (#18508)
Closes #18477
2023-03-30 11:47:33 -04:00
Bartek Iwańczuk
381f5801f9
Revert "refactor(ext/node): Use Deno.inspect (#17960)" (#18491)
This reverts commit a3529d0232.

This change made debugging Node tests very hard - `AssertionError` is
now printed as `[Circular *1]` giving no visibility what failed.

We need to align two implementations together and remove this one then.
2023-03-30 15:33:28 +00:00
David Sherret
e0429e2ad6
fix(repl): improve package.json support (#18497)
1. Fixes a cosmetic issue in the repl where it would display lsp warning
messages.
2. Lazily loads dependencies from the package.json on use.
3. Supports using bare specifiers from package.json in the REPL.

Closes #17929
Closes #18494
2023-03-30 10:43:16 -04:00
Yoshiya Hinosawa
3deade4b14
chore(ext/node): run node compat parallel tests in core number concurrency (#18505)
We currently run the all test cases in `parallel` category at the same
time, which invokes hundreds process at the same time, and that seems
causing some flakiness in CI. (maybe related to #18487)

This PR limits the concurrency to the number of cpu cores. This is more
aligned to how Node.js run their `parallel` test in their repository.
42c4a35952/Makefile (L356)
2023-03-30 16:35:45 +02:00
Bartek Iwańczuk
d418f792a9
test: fix test_check_origin_not_supported (#18504)
Merge of dotland and dotcom caused this test to fail.
2023-03-30 19:17:39 +05:30
Bartek Iwańczuk
913e2875c1
refactor(ext/node): add NodeEnv::Fs associated type (#18484)
This commit adds associated type to "NodeEnv" trait, called "Fs".

The "Fs" type has a trait bound on "NodeFs", which specifies APIs
required for all ops and resolution APIs to function.

A "RealFs" implementation of "NodeFs" is exported from the "deno_node"
crate, that provides a default implementation for the trait.

All code in "deno_node" extension was changed to use the "NodeFs" trait
to handle file system operations, instead of relying on APIs from the
standard library.
2023-03-30 03:20:31 +02:00
David Sherret
89bbbd102c
refactor(lsp): remove boolean parameters on documents.documents(...) (#18493)
I think this makes things clearer at the call sites.
2023-03-29 16:25:48 -04:00
Andrew Nester
bacbf94925
test(ext/node): add timers_tests.ts from std/node (#18472) 2023-03-29 18:34:37 +09:00
Andrew Nester
04399d138e
test(ext/node): add string_decoder_test.ts from std/node (#18473) 2023-03-29 18:32:29 +09:00
David Sherret
b5d3eb5c23
chore: fix flaky pty_internal_repl (#18486) 2023-03-29 00:25:28 +00:00
Bartek Iwańczuk
ceab7a8d23
test(lsp): make lsp_completions_auto_import more robust (#18482)
A completely unrelated change in
https://github.com/denoland/deno/pull/18286 threw of this test.

These changes make it more robust and easier to update in such cases.
2023-03-28 23:58:12 +02:00
Bartek Iwańczuk
c5302a0587
refactor(ext/node): change extension type parameter (#18483)
This commit changes the type parameter for "deno_node" extension, from
`P: NodePermission` to `Env: NodeEnv`.

`NodeEnv` is a new trait that has associated type `P: NodePermission`.

This is a stepping stone to support swappable file system for the
extension, that will be added as a second associated type to the 
`NodeEnv` trait.
2023-03-28 23:52:08 +02:00
David Sherret
6fb6b0c1f3
chore: restore pty tests and make them run on the Linux CI (#18424)
1. Rewrites the tests to be more back and forth rather than getting the
output all at once (which I believe was causing the hangs on linux and
maybe mac)
2. Runs the pty tests on the linux ci.
3. Fixes a bunch of tests that were just wrong.
4. Adds timeouts on the pty tests.
2023-03-28 21:49:00 +00:00
Matt Mastracci
c65149c0a0
fix(core): restore cache journal mode to TRUNCATE and tweak tokio test in CacheDB (#18469)
Fast-follow on #18401 -- the reason that some tests were panicking in
the `CacheDB` `impl Drop` was that the cache itself was being dropped
during panic and the runtime may or may not still exist at that point.
We can reduce the actual tokio runtime testing to where it's needed.

In addition, we return the journal mode to `TRUNCATE` to avoid the risk
of data corruption.
2023-03-28 14:06:57 -06:00
Ryan Dahl
35fd8583ef
refactor: move shared library tests to their own file (#18479) 2023-03-28 13:40:43 -04:00
Kamal Singh
e789f813f9
test(node/fs): add fs.Dir tests (#18463)
Co-authored-by: Yoshiya Hinosawa <stibium121@gmail.com>
2023-03-28 23:58:40 +09:00
Yoshiya Hinosawa
4358ab2d84
fix(ext/node): implement crypto.Sign (RSA/PEM/SHA{224,256,384,512}) (#18471) 2023-03-28 21:46:48 +09:00
Divy Srivastava
10012c2fe3
feat(ext/node): add crypto.checkPrime API (#18465)
Towards #18455 

This commit implements `checkPrimeSync` and `checkPrime` in node:crypto
using the Miller-Rabin primality test (fun fact: it actually is a test
for composite numbers)

It first compares the candidate against many known small primes and if
not, proceeds to run the Miller-Rabin primality test.
http://nickle.org/examples/miller-rabin.5c used as reference
implementation.
2023-03-28 10:56:38 +00:00
Divy Srivastava
67e21e71ce
chore(ext/node): port pbkdf2 to Rust (#18470)
Towards #18455
2023-03-28 09:40:56 +00:00
Andrew Nester
d0a0ff680e
chore: add test for Linux shared libraries (#18461)
Closes #18266
2023-03-28 09:00:19 +00:00
Matt Mastracci
86c3c4f343
feat(core): initialize SQLite off-main-thread (#18401)
This gets SQLite off the flamegraph and reduces initialization time by
somewhere between 0.2ms and 0.5ms. In addition, I took the opportunity
to move all the cache management code to a single place and reduce
duplication. While the PR has a net gain of lines, much of that is just
being a bit more deliberate with how we're recovering from errors.

The existing caches had various policies for dealing with cache
corruption, so I've unified them and tried to isolate the decisions we
make for recovery in a single place (see `open_connection` in
`CacheDB`). The policy I chose was:

 1. Retry twice to open on-disk caches
 2. If that fails, try to delete the file and recreate it on-disk
3. If we fail to delete the file or re-create a new cache, use a
fallback strategy that can be chosen per-cache: InMemory (temporary
cache for the process run), BlackHole (ignore writes, return empty
reads), or Error (fail on every operation).

The caches all use the same general code now, and share the cache
failure recovery policy.

In addition, it cleans up a TODO in the `NodeAnalysisCache`.
2023-03-27 22:01:52 +00:00
Marvin Hagemeister
8c051dbd1a
fix(ext/node): add missing _preloadModules hook (#18447)
This internal node hook is used by libraries such as `ts-node` when used
as a require hook `node -r ts-node/register`. That combination is often
used with test frameworks like `mocha` or `jasmine`.

We had a reference to `Module._preloadModules` in our code, but the
implementation was missing. While fixing this I also noticed that the
`fakeParent` module that we create internally always threw because of
the `pathDirname` check on the module id in the constructor of `Mdoule`.
So this code path was probably broken for a while.

```txt
✖ ERROR: Error: Empty filepath.
    at pathDirname (ext:deno_node/01_require.js:245:11)
    at new Module (ext:deno_node/01_require.js:446:15)
    at Function.Module._resolveFilename (ext:deno_node/01_require.js:754:28)
    at Function.resolve (ext:deno_node/01_require.js:1015:19)
```
2023-03-27 21:54:22 +02:00
Bartek Iwańczuk
101abf3541
refactor(cli): remove Lazy<String> and fix help outputs (#18462)
This `Lazy<String>` calls were showing up on flamegraph and there's
really no point in using them.
2023-03-27 18:28:41 +02:00
Divy Srivastava
9ebce6e725
fix(cli/bench): look for clone3 syscalls for thread count (#18456) 2023-03-27 14:45:00 +00:00
Marvin Hagemeister
355275ec0f
fix(cli): add colors to "Module not found" error frame (#18437) 2023-03-27 01:10:47 +02:00
Geert-Jan Zwiers
a29d88b43b
feat(bench): add --no-run flag (#18433) 2023-03-26 14:55:58 +00:00
Max Dahlgren
701099b2a9
test(ext/node): Port crypto tests from std/node (#18382) 2023-03-26 10:03:10 +00:00
Nayeem Rahman
b4c61c146a
fix(cli): don't store blob and data urls in the module cache (#18261) 2023-03-26 12:24:10 +03:00
滑威
0742ea1170
fix(cli): deno upgrade file permission (#18427) 2023-03-26 12:19:12 +03:00
Bartek Iwańczuk
4d09b0c294
refactor: use default implementation of BootstrapOptions (#18439)
Drive-by cleanup while I was looking into serialization of
BootstrapOptions.
There's no need to use non-default implementation in these places.
2023-03-26 04:38:45 +00:00
Leo Kettmeir
33362b88c3
chore: upgrade clap to v4 (#17333) 2023-03-26 06:06:18 +02:00
Nayeem Rahman
8a4865c379
feat(test): print pending tests on sigint (#18246) 2023-03-25 21:32:11 +02:00
Leo Kettmeir
fe88b53e50
refactor: include mitata (#18426) 2023-03-25 15:29:46 +01:00
Geert-Jan Zwiers
e4c60bc1c6
chore(tests): enable single_compile_with_reload again (#18196) 2023-03-25 09:59:39 +00:00
Heyang Zhou
27834dfc10
chore(ext/kv): add limits (#18415) 2023-03-25 15:29:36 +08:00
Yoshiya Hinosawa
70e2e8f2dd
fix(ext/node): add aes-128-ecb algorithm support (#18412) 2023-03-25 15:42:07 +09:00
David Sherret
675179a176
chore(types): fix type checking errors in 99_main_compiler.js (#18403) 2023-03-24 10:35:44 -04:00
Divy Srivastava
d740a9e43d
feat(ext/node): implement crypto.createSecretKey (#18413)
This commit adds the `crypto.createSecretKey` API.

Key management: This follows the same approach as our WebCrypto
CryptoKey impl where we use WeakMap for storing key material and a
handle is passed around, such that (only internal) JS can access the key
material and we don't have to explicitly close a Rust resource.

As a result, `createHmac` now accepts a secret KeyObject.

Closes https://github.com/denoland/deno/issues/17844
2023-03-24 14:13:26 +00:00
Yoshiya Hinosawa
3d75fb2be7
fix(ext/node): make cipher/decipher transform stream (#18408) 2023-03-24 22:29:14 +09:00
Heyang Zhou
94ef428b56
fix(ext/kv): add missing getMany method (#18410)
The `getMany` method was missing from the implementation of the
`Deno.Kv` class. This patch fixes it.
2023-03-24 20:06:27 +08:00
Bartek Iwańczuk
275dee60e7
refactor: make version and user_agent &'static str (#18400)
These caused a bunch of unnecessary allocations on each startup.
2023-03-23 23:27:58 +01:00
Matt Mastracci
ad77ba0f7b
fix(core): panic at build time if extension code contains anything other than 7-bit ASCII (#18372)
This will improve diagnostics and catch any non-ASCII extension code
early.

This will use `debug_assert!` rather than `assert!` to avoid runtime
costs, and ensures (in debug_assert mode only) that all extension source
files are ASCII as we load them.
2023-03-23 16:00:46 -06:00
denobot
e8348231df
chore: forward v1.32.1 release commit to main (#18399)
Co-authored-by: bartlomieju <bartlomieju@users.noreply.github.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-03-23 22:55:23 +01:00
Cre3per
eb25e50edb
fix(cli): restore deno run - to handle stdin as typescript (#18391)
Bug reported here shortly after merging `--ext` changes
https://github.com/denoland/deno/pull/17172#issuecomment-1480898098

Also found a missing `--check` in integration tests for `--ext` that
would have missed a bug if there was one.

Fixes #18392
2023-03-23 12:45:43 -04:00
David Sherret
81c5ddf9f2
fix(inspect): ensure non-compact output when object literal has newline in entry text (#18366)
Fixes `Deno.inspect` to make an object literal non-compact when an entry
has multiple lines in it.
2023-03-23 10:58:53 -04:00
David Sherret
6e5a631fe0
refactor(lsp): add LspClientUrl (#18376)
This will make it a bit harder to accidentally use a client url in the
wrong place. I don't fully understand why we do this mapping, but this
will help prevent bugs like #18373

Closes #18374
2023-03-23 14:23:04 +00:00
Ryan Dahl
a3529d0232
refactor(ext/node): Use Deno.inspect (#17960)
No need for two almost identical implementations of the same thing

---------

Co-authored-by: Yoshiya Hinosawa <stibium121@gmail.com>
Co-authored-by: Aapo Alasuutari <aapo.alasuutari@gmail.com>
2023-03-23 10:01:07 -04:00
Bartek Iwańczuk
2117d9c1a7
refactor(init): remove CURRENT_STD_URL (#18375)
There's no point in having `Lazy<Url>`, since the only use case 
is string substitution in the "deno init" subcommand.
2023-03-23 03:11:18 +00:00
David Sherret
64f491422b
fix(lsp): ensure enablePaths works when clients do not provide a trailing slash for workspace dir (#18373)
Closes https://github.com/denoland/vscode_deno/issues/827
2023-03-23 01:58:38 +00:00
denobot
25b564bf86
1.32.0 (#18367)
Bumped versions for 1.32.0

---------

Co-authored-by: mmastrac <mmastrac@users.noreply.github.com>
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: David Sherret <dsherret@users.noreply.github.com>
2023-03-22 22:22:24 +00:00
Heyang Zhou
533e33131f
fix(ext/kv): reverse mapping between AnyValue::Bool and KeyPart::Bool (#18365)
Previously the mapping between `AnyValue::Bool` and `KeyPart::Bool` was
inverted.

This patch also allows using the empty key `[]` as range start/end to
`snapshot_read`.
2023-03-22 21:53:16 +01:00
Heyang Zhou
a117d3d91e
chore(ext/kv): disable ci tests on darwin (#18364)
https://github.com/denoland/deno/issues/18363
2023-03-22 20:46:26 +01:00
Ryan Dahl
f9c8d98b77
Revert "refactor: rename Deno.openKv() to Deno.kv() (#18349)" (#18362)
This reverts commit 50b793c9ed.
2023-03-22 20:23:36 +01:00
Matt Mastracci
cebefa8783
chore(test_util): replace tempdir code w/tempdir crate (#18340) 2023-03-22 12:55:19 -06:00
Bartek Iwańczuk
df614ff6e5
test: ignore flaky node:child_process tests (#18360) 2023-03-22 18:39:13 +00:00
Matt Mastracci
03e236b614
chore(cli): measure performance against wall-clock (#18357)
Fixes test flake by requiring perf to be <150% of wall-clock time rather
than <500ms.
2023-03-22 18:06:29 +00:00
Matt Mastracci
79fb3b1f35
chore(cli): ensure no signal on test exit (#18354)
If deno crashes on exit, we get a failure on the exit code (None instead
of Some(0) but we never see the signal.
2023-03-22 18:00:07 +00:00
David Sherret
1f635b1eac
fix: include error in message about not being able to create the TypeScript cache (#18356) 2023-03-22 13:59:55 -04:00
Matt Mastracci
42d13c4e6a
chore(cli): disable flash_test as it is causing SIGSEGV (#18358) 2023-03-22 13:42:50 -04:00
Bartek Iwańczuk
2ac2906b2b
test: ignore fetchWithInvalidContentLength and fetchConnectionError on Windows (#18351)
https://github.com/denoland/deno/issues/18350
2023-03-22 10:19:11 -04:00
Cre3per
fd0658fb42
feat(cli): --ext parameter for run, compile, and bundle (#17172)
Adds `--ext` to `deno run`, closes #5088

Additionally

- Adds `--ext` to `deno compile` and `deno bundle`
2023-03-22 10:15:53 -04:00
Ryan Dahl
50b793c9ed
refactor: rename Deno.openKv() to Deno.kv() (#18349) 2023-03-22 10:02:40 -04:00
Bartek Iwańczuk
e73e8410f6
chore: upgrade rusty_v8 to 0.66.0 (#18339) 2023-03-22 13:38:56 +01:00
Bartek Iwańczuk
aa7f02946d
test: mark two unit tests as flaky (#18344)
Temporarily marking two "fetch" tests as flaky, since
they've been failing on CI for the past 24h.
2023-03-22 13:30:54 +01:00
Nayeem Rahman
5804d7434e
fix(ext/kv): don't request permissions for ":memory:" (#18346)
Currently `Deno.openKv(":memory:")` requests read+write permissions for
`./:memory:` even though no file is read or written. Also added some
guards for special sqlite paths that were unintentionally opted into.
2023-03-22 06:49:29 -04:00
Heyang Zhou
92ebf4afe5
feat(ext/kv): key-value store (#18232)
This commit adds unstable "Deno.openKv()" API that allows to open
a key-value database at a specified path.

---------

Co-authored-by: Luca Casonato <hello@lcas.dev>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-03-22 12:13:24 +08:00
Bartek Iwańczuk
8bcffff9dc
test: disable macos_shared_libraries test (#18342)
This test is flaky: https://github.com/denoland/deno/issues/18341
2023-03-21 18:23:35 -06:00
Bartek Iwańczuk
619806d7a9
perf: disable WAL for transpiled source cache (#18084)
Disable Write-Ahead Log for the cached module source database.

This brings SQLite connection cost on startup from 2.5% to 1.6%.
2023-03-22 00:01:15 +00:00
Bartek Iwańczuk
7d9653d51f
Revert "fix(cli) Better error messages on corrupt databases that aren… (#18337)
…'t automatically re-created (#18330)"

This reverts commit 2ef8269fdb.

Printing these messages by default (instead of requiring `-L debug`
flag) caused
various tests to start printing it and mismatch in output assertions.
2023-03-21 22:38:18 +00:00
Matt Mastracci
0b4770fa7d
perf(core) Reduce script name and script code copies (#18298)
Reduce the number of copies and allocations of script code by carrying
around ownership/reference information from creation time.

As an advantage, this allows us to maintain the identity of `&'static
str`-based scripts and use v8's external 1-byte strings (to avoid
incorrectly passing non-ASCII strings, debug `assert!`s gate all string
reference paths).

Benchmark results:

Perf improvements -- ~0.1 - 0.2ms faster, but should reduce garbage
w/external strings and reduces data copies overall. May also unlock some
more interesting optimizations in the future.

This requires adding some generics to functions, but manual
monomorphization has been applied (outer/inner function) to avoid code
bloat.
2023-03-21 22:33:12 +00:00
David Sherret
253b556e6f
perf(check): type check local files only when not using --all (#18329)
Closes #18171
2023-03-21 18:19:42 -04:00
Bartek Iwańczuk
7e61e8f0e0
chore: update formatting configuration (#18331) 2023-03-21 17:01:53 -04:00
Bartek Iwańczuk
cc46f6776c
Revert "perf: disable TSC snapshot compression (#18333)" (#18336)
This reverts commit 0d27de943a.
2023-03-21 19:11:22 +00:00
Matt Mastracci
a561cc28cd
chore(cli) Use with_context(|| format!(...)) rather than context(format!(...)) to avoid allocations in non-error path (#18332) 2023-03-21 12:13:32 -06:00
Bartek Iwańczuk
0d27de943a
perf: disable TSC snapshot compression (#18333)
This commit disables compression of the TSC snapshot.
The compression only decreased the size of snapshot by 0.5Mb
and it took about 40s during release build to compress.

With recent gains in TS 5.0 upgrade in terms of size and performance
it makes sense to remove this compression.
2023-03-21 17:53:46 +00:00
Matt Mastracci
2ef8269fdb
fix(cli) Better error messages on corrupt databases that aren't automatically re-created (#18330) 2023-03-21 17:19:10 +00:00
David Sherret
2fcf1f14cf
feat: TypeScript 5.0.2 (except decorators) (#18294)
This upgrades TypeScript to 5.0.2, but does not have ES decorator
support because swc does not support that yet.
2023-03-21 15:46:40 +00:00
Yoshiya Hinosawa
aa729a42b4
chore(tools): restore node compat test setup script (#18290) 2023-03-21 22:38:07 +09:00
ayame113
c34e26a9d5
docs: fix broken Deno.FsFile#readable example code (#18277) 2023-03-21 16:22:21 +09:00
Yoshiya Hinosawa
896d7f02cb
chore(ext/console): inspect anonymous function as [Function (anonymous)] (#18283)
This PR changes the inspect result of anonymous functions from
`[Function]` to `[Function (anonymous)]`. This behavior is aligned
to `util.inspect` of Node.js.
2023-03-21 13:12:13 +09:00
Sam Gwilym
4c34a2f2df
feat(ext/net): Add multicasting APIs to DatagramConn (#10706) (#17811) 2023-03-20 22:27:00 +01:00
Bartek Iwańczuk
cd53ab5427
refactor(ext/node): untangle dependencies between js files (#18284)
Moving some code around in `ext/node` is it's a bit better well defined
and makes it possible for others to embed it.

I expect to see no difference in startup perf with this change.
2023-03-20 14:05:13 -04:00
Bartek Iwańczuk
aba5329aec
refactor(ext/node): make initialization functions sync (#18282)
These functions don't need to be async, as they are only calling
synchronous JavaScript code. As a follow up, all 3 functions
should be merge together - this will reduce roundtrips for
calling V8 from Rust, which is somewhat expensive
2023-03-20 00:40:21 -04:00
Andreu Botella
090169cfbc
feat(compile): Add support for web workers in standalone mode (#17657)
This commit adds support for spawning Web Workers in self-contained
binaries created with "deno compile" subcommand.

As long as module requested in "new Worker" constructor is part of the
eszip (by means of statically importing it beforehand, or using "--include"
flag), then the worker can be spawned.
2023-03-19 23:32:54 +01:00
Max Dahlgren
bf149d047f
test(ext/node): port _fs tests from std/node (#18262)
Co-authored-by: Yoshiya Hinosawa <stibium121@gmail.com>
2023-03-20 01:10:39 +09:00
Sanskar Gauchan
3b1cb8af69
test(ext/node): add querystring_test.ts and readline_test.ts (#18256) 2023-03-19 19:49:11 +09:00
Sanskar Gauchan
472c06b92e
test(ext/node): add _fs_access_test.ts and _fs_watch_test.ts (#18249)
part of https://github.com/denoland/deno/issues/17840
2023-03-19 19:37:00 +09:00
Kamil Ogórek
dd1e650063
fix(runtime): Extract error code for all OS error variants (#17958) 2023-03-19 00:01:50 +00:00
Andreu Botella
b64ec79268
feat(compile): Enable multiple roots for a standalone module graph (#17663)
This change will enable dynamic imports and web workers to use modules
not reachable from the main module, by passing a list of extra side
module roots as options to `deno compile`. 

This can be done by specifying "--include" flag that accepts a file path or a
URL. This flag can be specified multiple times, to include several modules.
The modules specified with "--include" flag, will be added to the produced
"eszip".
2023-03-19 00:43:07 +01:00
Bartek Iwańczuk
3c9771deb2
Reland "perf(core): preserve ops between snapshots (#18080)" (#18272)
Relanding 4b6305f4f2
2023-03-18 18:30:04 -04:00
Bartek Iwańczuk
d33369c6c7
perf(core): use static specifier in ExtensionFileSource (#18271)
Relanding #18264

-------

Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
2023-03-18 20:09:13 +00:00
Leo Kettmeir
4c1f3daa83
chore(bench): add RUNTIME_SNAPSHOT.bin (#18269) 2023-03-18 21:00:00 +01:00
Bartek Iwańczuk
f6c364fcf6
Revert "perf(core): use static specifier in ExtensionFileSource (#182… (#18270)
…64)"

This reverts commit 8af0c83519.

Causes bench stage to fail on CI.
2023-03-18 20:25:28 +01:00
Divy Srivastava
00f4c0df97
chore(cli/tests): move macos_shared_libraries test to macos_test.rs (#18268)
As per review comment in original PR:
https://github.com/denoland/deno/pull/18244#issuecomment-1473769606
2023-03-18 21:08:51 +05:30
Bartek Iwańczuk
d11e89127d
Revert "perf(core): preserve ops between snapshots (#18080)" (#18267)
This reverts commit 4b6305f4f2.
2023-03-18 14:59:51 +00:00
Divy Srivastava
8af0c83519
perf(core): use static specifier in ExtensionFileSource (#18264)
Closes #18085

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-03-18 13:14:28 +00:00
Yoshiya Hinosawa
5223f3d9b6
fix(ext/node): add createDecipheriv (#18245) 2023-03-18 21:51:28 +09:00
Andrew
44553aa09e
docs(FFI/UnsafePointerView): fix a typo in a docstring (#18034) 2023-03-18 12:25:06 +00:00
Bartek Iwańczuk
4b6305f4f2
perf(core): preserve ops between snapshots (#18080)
This commit changes the build process in a way that preserves already
registered ops in the snapshot. This allows us to skip creating hundreds of
"v8::String" on each startup, but sadly there is still some op registration
going on startup (however we're registering 49 ops instead of >200 ops). 

This situation could be further improved, by moving some of the ops 
from "runtime/" to a separate extension crates.

---------

Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
2023-03-18 12:51:21 +01:00
Geert-Jan Zwiers
058865c9a3
fix(cli): preserve blob store when resetting file watcher (#18253)
Fixes https://github.com/denoland/deno/issues/17733

---------

Co-authored-by: Matt Mastracci <matthew@mastracci.com>
2023-03-17 19:58:25 -06:00
Matt Mastracci
3487fde236
perf(core) Reduce copying and cloning in extension initialization (#18252)
Follow-up to #18210:

* we are passing the generated `cfg` object into the state function
rather than passing individual config fields
 * reduce cloning dramatically by making the state_fn `FnOnce`
 * `take` for `ExtensionBuilder` to avoid more unnecessary copies
 * renamed `config` to `options`
2023-03-17 22:15:27 +00:00
Matt Mastracci
e55b448730
feat(core) deno_core::extension! macro to simplify extension registration (#18210)
This implements two macros to simplify extension registration and centralize a lot of the boilerplate as a base for future improvements:

* `deno_core::ops!` registers a block of `#[op]`s, optionally with type
parameters, useful for places where we share lists of ops
* `deno_core::extension!` is used to register an extension, and creates
two methods that can be used at runtime/snapshot generation time:
`init_ops` and `init_ops_and_esm`.

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-03-17 18:22:15 +00:00
Bartek Iwańczuk
0bc6bf5d33
chore: add tests for node:async_hooks (#18004)
Closes https://github.com/denoland/deno/issues/17878

---------

Co-authored-by: crowlkats <crowlkats@toaxl.com>
2023-03-17 18:53:57 +01:00
Farsen976
e6fe1630f0
fix(cli/integration): clippy lints (#18248)
This addresses two warnings from clippy, redundant-static-lifetimes and trim-split-whitespace.
closes #18247
2023-03-17 20:51:32 +09:00
Divy Srivastava
0eabd2c6d6
chore: add test for macOS shared libraries (#18244)
Closes https://github.com/denoland/deno/issues/18243
2023-03-17 07:41:17 +02:00
Yoshiya Hinosawa
e30d24be72
chore(ext/console): add 'quotes' internal option to Deno.inspect (#18183) 2023-03-17 12:22:09 +09:00
Bartek Iwańczuk
8f9becee76
perf: disable runtime snapshot compression (#18239)
This commit removes compression for the runtime JS code.

It means that we will have a bigger binary, but faster startup. After
several discussion in the CLI team we decided it's worth to trade
about 3Mb of binary size for 2ms faster startup time. With WebGPU
removed in 35196eab27
it shouldn't have such a big impact on the binary size.
2023-03-17 08:49:46 +05:30
Leo Kettmeir
35196eab27
BREAKING(unstable): remove WebGPU (#18094)
This PR _**temporarily**_ removes WebGPU (which has behind the
`--unstable` flag in Deno), due to performance complications due to its
presence.

It will be brought back in the future; as a point of reference, Chrome
will ship WebGPU to stable on 26/04/2023.

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-03-17 00:29:32 +01:00
Bartek Iwańczuk
3f031ad9af
BREAKING(ext/fs): FileInfo.dev is defined on Windows (#18237)
Addresses feedback from
https://github.com/denoland/deno/pull/18073#issuecomment-1471480385.

Reverts changes to `FileInfo` fields that are not available on Windows
making them `null`. Only `FileInfo.dev` is non-null.
2023-03-17 00:14:56 +01:00
Divy Srivastava
1300d6178e
fix(ext/node): resource leak in createHmac (#18229)
This commit fixes https://github.com/denoland/deno/issues/18140.
Verified that test fails on `main`.
2023-03-16 22:25:12 +01:00
Roy Li
2c7174a5a2
fix(repl): Hide indexable properties in tab completion (#18141)
Closes #17831. This change hides the indices of any indexed collection
when triggering tab completion for object properties in the REPL.

An example is shown in the issue, but for verbosity here is another.

Before the change:
```
> const arr = new Uint8ClampedArray([1, 2, 3])
undefined
> arr.
0                     map
1                     reverse
2                     reduce
...
```
After the change:
```
> const arr = new Uint8ClampedArray([1, 2, 3])
undefined
> arr.
constructor               reduce
BYTES_PER_ELEMENT         reduceRight
buffer                    set
...
```

Co-authored-by: David Sherret <dsherret@users.noreply.github.com>
2023-03-16 19:46:50 +00:00
David Sherret
56731f2f8d
chore: ignore permission_prompt_strips_ansi_codes_and_control_chars test (#18234)
This test hangs on all operating systems.

Opened https://github.com/denoland/deno/issues/18233 for the future
2023-03-16 14:36:23 -04:00
Bartek Iwańczuk
29c9a5c90d
refactor: reorder op initialization (#18228)
To be able to preserve "Deno.core.ops" we need to ensure that
ops are registered in the same order in various places, otherwise
we will get mismatch in external references ordering.

Prerequisite for https://github.com/denoland/deno/pull/18080
2023-03-16 18:36:53 +01:00
Nick Hanley
1a3c2e2f1d
feat(repl): add DENO_REPL_HISTORY to change history file path (#18047) 2023-03-17 01:22:24 +09:00
denobot
33b85a2b8e
chore: forward v1.31.3 release commit to main (#18222)
Co-authored-by: kt3k <kt3k@users.noreply.github.com>
2023-03-16 17:09:26 +09:00
Farsen976
96bc15dfa2
fix(ext/node): implement "ascii" encoding for node:fs writeFile() (#18097) 2023-03-16 12:16:03 +09:00
David Sherret
c1eba16b84
fix(repl): do not panic deleting Deno or deleting all its properties (#18211)
Closes #18194
Closes #12092
2023-03-15 21:41:13 -04:00
Bartek Iwańczuk
48a0b7f98f
feat(fs): support FileInfo.dev on Windows (#18073)
This commit adds support for retrieving `dev` information
when stating files on Windows.

Additionally `Deno.FileInfo` interfaces was changed to always
return 0 for fields that we don't retrieve information for on Windows.

Closes https://github.com/denoland/deno/issues/18053

---------

Co-authored-by: David Sherret <dsherret@gmail.com>
2023-03-16 02:35:13 +01:00
Timo Wilhelm
92c3ac3034
fix(ext/http): abort request signal when response errors (#17822) 2023-03-15 22:37:06 +00:00
David Sherret
fb021d7cef
refactor: remove usages of map_or / map_or_else (#18212)
These methods are confusing because the arguments are backwards. I feel
like they should have never been added to `Option<T>` and that clippy
should suggest rewriting to
`map(...).unwrap_or(...)`/`map(...).unwrap_or_else(|| ...)`

https://github.com/rust-lang/rfcs/issues/1025
2023-03-15 17:46:36 -04:00
David Sherret
ca51f4f6c0
fix(npm): "not implemented scheme" message should properly show the scheme (#18209) 2023-03-15 15:23:30 -04:00
Divy Srivastava
3a46a89e34
chore: add test for ws ping/pong (#18204)
This commit adds test for https://github.com/denoland/deno/issues/17761
which was fixed by https://github.com/denoland/deno/pull/17762. Verified
that test fails on Deno 1.30.1
2023-03-15 22:09:09 +05:30
David Sherret
5c505316b3
chore(tests): fix flaky test_no_lock (#18206)
The test output capturing is known to be flaky.

Closes #18199
2023-03-16 00:03:36 +09:00
David Sherret
7070b8ed50
fix(lsp): avoid calling client while holding lock (#18197) 2023-03-15 14:34:23 +00:00
Divy Srivastava
51649272bd
perf: do not depend on iana-time-zone (#18088)
Chrono's `clock` feature pulls in `iana-time-zone` which links to macOS
core_foundation. This PR itself is not enough to get rid of
CoreFoundation. Removal depends on getting rid of security framework,
see #18071
2023-03-15 07:14:22 +00:00
Bartek Iwańczuk
96ea5b1d3d
refactor: deno_core not using std::env::current_dir (#18173)
This commit changes "deno_core" to not rely on implicitly calling
"std::env::current_dir()" when resolving module specifiers using
APIs from "deno_core::modules_specifier".

Supersedes https://github.com/denoland/deno/pull/15454
2023-03-14 16:47:09 -04:00
Bartek Iwańczuk
1930c09b04
refactor(core): remove "resolve_url_or_path_deprecated" (#18174)
Remove remaining usages of "resolve_url_or_path_deprecated" in favor
of "resolve_url_or_path" with explicit calls to
"std::env::current_dir()".

Towards landing https://github.com/denoland/deno/pull/15454
2023-03-14 17:18:01 +00:00
Yiyu Lin
96b1ede254
refactor(lockfile): move anyhow to thiserror (#18178) 2023-03-14 10:55:52 -04:00
Bartek Iwańczuk
6f308b3af0
test: fix flaky resolve_import_map_flags_take_precedence test (#18182)
Trying to fix https://github.com/denoland/deno/issues/18180
2023-03-14 13:37:15 +02:00
Nayeem Rahman
c47075ba4d
test: parallelize applicable node compat tests (#18161) 2023-03-14 16:56:06 +09:00
Yoshiya Hinosawa
e80cc17dc4
fix(ext/node): add crypto.createCipheriv (#18091) 2023-03-14 15:59:23 +09:00
Bartek Iwańczuk
9aa20b3ba7
refactor: --watch commands use deno_core::resolve_url_or_path (#18172)
This commit changes various CLI subcommands that have support for
the "--watch" flag to use initial current working directory when
resolving "main module". 

This is part of migration towards explicitly passing current working
directory to "deno_core::resolve_url_or_path" API.

As a side effect this makes "deno <subcommand> --watch" more aligned to
user expectations, where calling "Deno.chdir()" during program doesn't
break watcher.

Towards landing https://github.com/denoland/deno/pull/15454
2023-03-14 02:44:16 +00:00
Bartek Iwańczuk
48ede89f1f
refactor(core): resolve_url_or_path and resolve_url_or_path_deprecated (#18170)
This commit changes current "deno_core::resolve_url_or_path" API to
"resolve_url_or_path_deprecated" and adds new "resolve_url_or_path"
API that requires to explicitly pass the directory from which paths
should be resolved to. 

Some of the call sites were updated to use the new API, the reminder
of them will be updated in a follow up PR.

Towards landing https://github.com/denoland/deno/pull/15454
2023-03-14 01:12:09 +00:00
Bartek Iwańczuk
c4771356f2
refactor: Remove call sites of "deno_core::resolve_url_or_path" (#18169)
These call sites didn't need to use "resolve_url_or_path".

Towards landing https://github.com/denoland/deno/pull/15454
2023-03-13 19:31:03 -04:00
David Sherret
a35c8e6588
fix(info/doc): add missing --no-lock and --lock flags (#18166)
Closes #18159
2023-03-13 17:04:00 -04:00
David Sherret
44b0d4cb11
fix(npm): show a progress bar when initializing the node_modules folder (#18136)
Creating the node_modules folder when the packages are already
downloaded can take a bit of time and not knowing what is going on can
be confusing. It's better to show a progress bar.
2023-03-13 18:18:29 +00:00
Bartek Iwańczuk
e8f22c0765
refactor(core): pass cwd explicitly to resolve_path (#18092)
Towards landing #15454
2023-03-13 18:50:23 +01:00
David Sherret
11f225ef7d
fix: ensure no node_modules directory is created when a package.json exists and no npm dependencies are used (#18134)
Closes #18133
Closes #18038
2023-03-13 14:03:19 +00:00
Geert-Jan Zwiers
983447e860
chore(cli/tests): use test builder in more integration tests (#18031) 2023-03-13 09:40:46 -04:00
Ryan Dahl
224cff7a2f
Revert "feat(core): prevent isolate drop for CLI main worker (#18059)" (#18157)
Fixes https://github.com/denoland/deno/issues/18120
https://github.com/denoland/deno/issues/18137
https://github.com/denoland/fresh/issues/1073

This reverts commit 0cce9c2bcc.
2023-03-13 12:58:17 +00:00
Kenta Moriuchi
e5673f5ed8
fix(core): SafePromiseAll to be unaffected by Array#@@iterator (#17542) 2023-03-13 19:24:31 +09:00
David Sherret
bcb6ee9d08
refactor(npm): push npm struct creation to a higher level (#18139)
This has been bothering me for a while and it became more painful while
working on #18136 because injecting the shared progress bar became very
verbose. Basically we should move the creation of all these npm structs
up to a higher level.

This is a stepping stone for a future refactor where we can improve how
we create all our structs.
2023-03-12 23:32:59 -04:00
David Sherret
8db853514c
fix(check): regression where config "types" entries caused type checking errors (#18124)
Closes #18117
Closes #18121 (this is just over 10ms faster in a directory one up from
the root folder)

cc @nayeemrmn
2023-03-11 11:43:45 -05:00
Ikko Eltociear Ashimine
e4430400ce
chore: typo (#18128) 2023-03-11 08:58:55 -05:00
denobot
857393d934
chore: forward v1.31.2 release commit to main (#18114)
Co-authored-by: Yoshiya Hinosawa <stibium121@gmail.com>
2023-03-10 17:39:29 +09:00
Lino Le Van
c7a5f928d3
chore(cli): fix outdated comment (#18106) 2023-03-09 21:13:53 -05:00
Bartek Iwańczuk
d1685b120b
refactor(core): remove RuntimeOptions::extensions_with_js (#18099)
This commit removes "deno_core::RuntimeOptions::extensions_with_js".

Now it's embedders' responsibility to properly register extensions
that will not contains JavaScript sources when running from an existing
snapshot.

Prerequisite for https://github.com/denoland/deno/pull/18080
2023-03-09 19:22:27 -05:00
Bartek Iwańczuk
78d430103a
fix(prompt): better output with control chars (#18108) 2023-03-10 01:09:14 +01:00
Bartek Iwańczuk
44e67df1cd
fix: Split extension registration and snapshotting (#18098)
This commit partially reverts changes from
https://github.com/denoland/deno/pull/18095.

Turns out I made a mistake that became apparent when working
on removing "RuntimeOptions::extensions_with_js" in a follow up.
2023-03-09 21:09:45 +01:00
David Sherret
47012bd931
refactor(tests/lsp): consolidate more into test LspClient and reduce verbosity (#18100) 2023-03-09 15:09:03 -05:00
Bartek Iwańczuk
bb07e230d1
chore: update Rust to 1.68.0 (#18102) 2023-03-09 19:18:00 +00:00
Bartek Iwańczuk
8f207c0f3f
refactor: Split extension registration for runtime and snapshotting (#18095)
This commit splits "<ext_name>::init" functions into "init_ops" and
"init_ops_and_esm". That way we don't have to construct list of
ESM sources on each startup if we're running with a snapshot.

In a follow up commit "deno_core" will be changed to not have a split
between "extensions" and "extensions_with_js" - it will be embedders'
responsibility to pass appropriately configured extensions.

Prerequisite for https://github.com/denoland/deno/pull/18080
2023-03-09 14:56:19 +00:00
Divy Srivastava
99da8a69e7
fix(ext/webstorage): check size of inputs before insert (#18087) 2023-03-09 12:28:51 +00:00
David Sherret
25d98ca289
refactor(lsp): improve test client initialization (#18015) 2023-03-08 23:15:20 +00:00
David Sherret
23e1ba7e2d
fix(npm): improve peer dependency resolution with circular dependencies (#18069)
This improves peer dependency resolution yet again. We did not handle
scenarios like the following:

```
// a -> b -> c -> d -> c -> b (peer)
```

...which would maybe work ok the first time its run in some cases, but
then lead to a lockfile that would error on load.

This now keeps track of circular dependencies and updates nodes
accordingly. That said, there is still a lurking bug in this code
somewhere that I've added a comment for (there is a mitigation on the
tail end that seems to work well). The current state is much better than
before and I can look into it later. I think it's something small that's
incorrect.
2023-03-08 12:22:08 -05:00
Divy Srivastava
0cce9c2bcc
feat(core): prevent isolate drop for CLI main worker (#18059)
```
Benchmark 1: deno run -A ../empty.js
  Time (mean ± σ):      20.5 ms ±   0.5 ms    [User: 13.4 ms, System: 5.1 ms]
  Range (min … max):    19.8 ms …  24.0 ms    119 runs

Benchmark 2: target/release/deno run -A ../empty.js
  Time (mean ± σ):      18.8 ms ±   0.3 ms    [User: 13.0 ms, System: 4.9 ms]
  Range (min … max):    18.3 ms …  19.9 ms    129 runs
  ```

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-03-08 22:21:14 +05:30
Bartek Iwańczuk
06a12341c6
chore: update tempfile dependency (#18077) 2023-03-08 16:46:27 +00:00
Bartek Iwańczuk
0e8fc4dc6c
Revert "perf: disable snapshot compression (#18061)" (#18074)
This reverts commit d12b039e1a.

It was landed as an experiment and it confirms much faster startup
if not compression is applied.
2023-03-08 21:14:04 +05:30
David Sherret
88b5fd9088
fix: attempt to only allow one deno process to update the node_modules folder at a time (#18058)
This is implemented in such a way that it should still allow processes
to go through when a file lock wasn't properly cleaned up and the OS
hasn't released it yet (but with a 200ms-ish delay).

Closes #18039
2023-03-08 15:13:13 +00:00
Bartek Iwańczuk
72fe9bb470
refactor: rename InternalModuleLoader to ExtModuleLoader, use ext: scheme for snapshotted modules (#18041)
This commit renames "deno_core::InternalModuleLoader" to
"ExtModuleLoader" and changes the specifiers used by the 
modules loaded from this loader to "ext:".

"internal:" scheme was really ambiguous and it's more characters than
"ext:", which should result in slightly smaller snapshot size.

Closes https://github.com/denoland/deno/issues/18020
2023-03-08 12:44:54 +01:00
Bartek Iwańczuk
d24c6ea27f
refactor(runtime): conditionally register Extension with source files (#18068)
Since we are snapshotting extension source at build time, there's no
need to define list of sources for the extension at runtime.

This commit changes "deno_node" extension by removing "init_polyfill"
function in favor of "init_polyfill_ops_and_esm()" and "init_polyfill_ops()".

The former is used during snapshot and when "deno_runtime" is compiled
with "dont_create_runtime_snapshot" cargo feature flag. The latter is used
when running a worker from an existing snapshot.

This is a start of a bigger refactor to all extensions - thanks to this
change, we don't have to iterate over all defined source files for extension at
runtime, and because of that we don't have to create a filepath for each of the
source files. It's not a big deal, but we are iterating over 300 files on each start,
and concatenating 3 strings before creating a "PathBuf" for ~200 of them.
This is already visible on the startup flamegraphs and should be avoided.
2023-03-08 12:43:26 +01:00
Bartek Iwańczuk
b32a6f8ad2
refactor(core): don't use Result in ExtensionBuilder::state (#18066)
There's no point for this API to expect result. If something fails it should
result in a panic during build time to signal to embedder that setup is
wrong.
2023-03-07 22:37:37 +01:00
Caleb Cox
0e3affcd5b
fix(cli): add space after period in --v8-flags (#18063)
Add space between period and "Any" in `--v8-flags` help text.
2023-03-07 17:07:28 +01:00
ayame113
08591d052a
chore(docs): Fixed manual link (#18060) 2023-03-07 08:01:56 -05:00
Bartek Iwańczuk
d12b039e1a
perf: disable snapshot compression (#18061)
This commit disables snapshot compression for the CLI snapshot.]

Decompressing the snapshot on startup takes ~2.5ms.
2023-03-07 13:57:58 +01:00
Bartek Iwańczuk
fe368b72c1
refactor: Add "deno_fs" extension crate (#18040)
This commit factors out APIs related to file system from "runtime/"
to a separate "deno_fs" extension crate.
2023-03-07 18:13:44 +09:00
Nick Hanley
64354f4112
chore(tests): use temp dir in test missing_deno_dir (#18057)
Fixes #18056
2023-03-07 00:00:51 -05:00
David Sherret
ff4b03f233
chore(tests): ability to capture stdout and stderr separately (#18035)
This is to allow making assertions on stdout and stderr separately.
2023-03-06 09:16:50 -05:00
Bartek Iwańczuk
5f34c9be91
refactor: move definition of Deno.build from "runtime" to "core" (#18036)
We use information about build in several extension crates like
"ext/node" or "runtime/". In an effort to move "fs" APIs to a separate 
crate it is a prerequisite to have this information available outside 
of the "runtime/" crate.

This commit moves definition of "build" object to "Deno.core" that is 
later forwarded to "Deno.build".
2023-03-05 22:18:13 +00:00
Nayeem Rahman
273777f7d9
fix(check): include dts files in tsc roots (#18026) 2023-03-05 12:47:04 -05:00
Filip Skokan
7d13d65468
fix(ext/crypto): correctly limit ECDSA and hash algorithms (#18030)
Closes #18029
2023-03-05 12:34:07 +00:00
Bartek Iwańczuk
888ceac7fd
refactor(runtime): remove 40_files.js, 40_write_file.js and 40_read_file.js (#18018)
JavaScript APIs from "runtime/js/40_files.js" combined abstractions
for stdio streams ("Stdout", "Stderr", "Stdin") and file system file
("File", "FsFile"). APIs from "runtime/js/40_read_file.js" and 
"runtime/js/40_write_file.js" were implemented using ops from 
"runtime/ops/fs.rs".

This file was removed and relevant APIs were moved to "deno_io/12_io.js"
and "runtime/js/30_fs.js".

This work is meant to enable factoring out "deno_fs" crate.
2023-03-05 03:37:37 +00:00
Bartek Iwańczuk
b40086fd7d
refactor(core): include_js_files! 'dir' option doesn't change specifiers (#18019)
This commit changes "include_js_files!" macro from "deno_core"
in a way that "dir" option doesn't cause specifiers to be rewritten 
to include it.

Example:
```
include_js_files! {
  dir "js",
  "hello.js",
}
```

The above definition required embedders to use:
`import ... from "internal:<ext_name>/js/hello.js"`. 
But with this change, the "js" directory in which the files are stored
is an implementation detail, which for embedders results in: 
`import ... from "internal:<ext_name>/hello.js"`.

The directory the files are stored in, is an implementation detail and 
in some cases might result in a significant size difference for the 
snapshot. As an example, in "deno_node" extension, we store the 
source code in "polyfills" directory; which resulted in each specifier 
to look like "internal:deno_node/polyfills/<module_name>", but with 
this change it's "internal:deno_node/<module_name>". 

Given that "deno_node" has over 100 files, many of them having 
several import specifiers to the same extension, this change removes
10 characters from each import specifier.
2023-03-05 02:31:38 +00:00
Andreu Botella
01028fcdf4
test(compile): Add a test for dynamic imports in deno compile (#18017)
denoland/eszip#115 added support for statically-analyzed dynamic imports
in eszip, which made `deno compile` support dynamic imports starting
from #17858. This PR adds a test for it.

----

This test is adapted from PR #17663.

Closes #17908
2023-03-05 01:37:54 +00:00
David Sherret
2f7222da8a
refactor: remove Semaphore::new(1) and use TaskQueue (#18014) 2023-03-04 20:07:11 -05:00