1
0
Fork 0
mirror of https://github.com/denoland/deno.git synced 2024-11-26 16:09:27 -05:00
Commit graph

10036 commits

Author SHA1 Message Date
Aapo Alasuutari
9d6584c16f
perf(ext/node): Optimise Buffer string operations (#20158)
Extracted from https://github.com/denoland/deno/pull/17815

Optimise Buffer's string operations, most significantly when dealing
with ASCII and UTF-16. Base64 and HEX encodings are affected to much
lesser degrees.

## Performance

### String length 15
With very small strings we're at break-even or sometimes even lose a tad
bit of performance from creating a `DataView` that ends up not paying
for itself.

**This PR:**
```
benchmark                                                     time (avg)        iter/s             (min … max)       p75       p99      p995
-------------------------------------------------------------------------------------------------------------- -----------------------------
Buffer.from ascii string                                       1.15 µs/iter     871,388.6   (728.78 ns … 1.56 µs)   1.23 µs   1.56 µs   1.56 µs
Buffer.from base64 string                                      1.63 µs/iter     612,790.9     (1.31 µs … 1.96 µs)   1.77 µs   1.96 µs   1.96 µs
Buffer.from utf16 string                                       1.41 µs/iter     707,396.3   (915.24 ns … 1.93 µs)   1.61 µs   1.93 µs   1.93 µs
Buffer.from hex string                                         1.87 µs/iter     535,357.9     (1.56 µs … 2.19 µs)      2 µs   2.19 µs   2.19 µs
Buffer.toString ascii string                                 154.58 ns/iter   6,469,162.8    (149.69 ns … 198 ns) 154.51 ns 182.89 ns 191.91 ns
Buffer.toString base64 string                                161.65 ns/iter   6,186,189.6 (150.91 ns … 181.15 ns) 165.18 ns 171.87 ns 174.94 ns
Buffer.toString utf16 string                                 292.74 ns/iter   3,415,959.8 (285.43 ns … 312.47 ns) 295.25 ns 310.47 ns 312.47 ns
Buffer.toString hex string                                    89.61 ns/iter  11,159,315.6  (81.09 ns … 123.77 ns)  91.09 ns 113.62 ns 119.28 ns
```

**Main:**
```
benchmark                                                     time (avg)        iter/s             (min … max)       p75       p99      p995
-------------------------------------------------------------------------------------------------------------- -----------------------------
Buffer.from ascii string                                       1.26 µs/iter     794,875.8     (1.07 µs … 1.46 µs)   1.31 µs   1.46 µs   1.46 µs
Buffer.from base64 string                                      1.65 µs/iter     607,853.3     (1.38 µs … 2.01 µs)   1.69 µs   2.01 µs   2.01 µs
Buffer.from utf16 string                                       1.34 µs/iter     744,894.6     (1.09 µs … 1.55 µs)   1.45 µs   1.55 µs   1.55 µs
Buffer.from hex string                                         2.01 µs/iter     496,345.8      (1.54 µs … 2.6 µs)   2.26 µs    2.6 µs    2.6 µs
Buffer.toString ascii string                                 150.16 ns/iter   6,659,630.5 (144.99 ns … 166.68 ns)  152.4 ns 157.26 ns 159.14 ns
Buffer.toString base64 string                                164.73 ns/iter   6,070,692.0 (158.77 ns … 185.63 ns) 168.48 ns 175.74 ns 176.68 ns
Buffer.toString utf16 string                                 150.61 ns/iter   6,639,864.0  (148.2 ns … 168.29 ns) 150.93 ns 157.21 ns 168.15 ns
Buffer.toString hex string                                    94.21 ns/iter  10,614,972.9   (86.21 ns … 98.75 ns)  95.43 ns  97.99 ns  98.21 ns
```

### String length 1500
With moderate lengths we already see great upsides for `Buffer.from()`
with ASCII and UTF-16.

**This PR:**
```
benchmark                                                     time (avg)        iter/s             (min … max)       p75       p99      p995
-------------------------------------------------------------------------------------------------------------- -----------------------------
Buffer.from ascii string                                       5.79 µs/iter     172,562.6     (4.72 µs … 4.71 ms)   5.04 µs   10.3 µs  11.67 µs
Buffer.from base64 string                                      5.08 µs/iter     196,678.9     (4.97 µs … 5.76 µs)   5.08 µs   5.76 µs   5.76 µs
Buffer.from utf16 string                                       9.68 µs/iter     103,316.5     (7.14 µs … 3.44 ms)  10.32 µs  13.42 µs  15.21 µs
Buffer.from hex string                                         53.7 µs/iter      18,620.2     (49.37 µs … 2.2 ms)  54.74 µs   72.2 µs  81.07 µs
Buffer.toString ascii string                                   6.63 µs/iter     150,761.3     (5.59 µs … 1.11 ms)   6.08 µs  15.68 µs  24.77 µs
Buffer.toString base64 string                                460.57 ns/iter   2,171,224.4 (448.33 ns … 511.73 ns) 465.05 ns 495.54 ns 511.73 ns
Buffer.toString utf16 string                                   6.52 µs/iter     153,287.0     (6.47 µs … 6.66 µs)   6.53 µs   6.66 µs   6.66 µs
Buffer.toString hex string                                     3.68 µs/iter     271,965.4     (3.64 µs … 3.82 µs)   3.68 µs   3.82 µs   3.82 µs
```

**Main:**
```
benchmark                                                     time (avg)        iter/s             (min … max)       p75       p99      p995
-------------------------------------------------------------------------------------------------------------- -----------------------------
Buffer.from ascii string                                      11.46 µs/iter      87,298.1    (8.53 µs … 834.1 µs)   9.61 µs  83.31 µs   87.3 µs
Buffer.from base64 string                                       5.4 µs/iter     185,027.8     (5.07 µs … 7.49 µs)   5.44 µs   7.49 µs   7.49 µs
Buffer.from utf16 string                                       20.3 µs/iter      49,270.8  (13.55 µs … 649.11 µs)   18.8 µs 113.93 µs 125.17 µs
Buffer.from hex string                                        52.03 µs/iter      19,218.9    (48.74 µs … 2.59 ms)  52.84 µs  67.05 µs  73.56 µs
Buffer.toString ascii string                                   6.46 µs/iter     154,822.5     (6.32 µs … 6.69 µs)   6.52 µs   6.69 µs   6.69 µs
Buffer.toString base64 string                                440.19 ns/iter   2,271,764.6    (427 ns … 490.77 ns) 444.74 ns 484.64 ns 490.77 ns
Buffer.toString utf16 string                                   6.89 µs/iter     145,106.7     (6.81 µs … 7.24 µs)   6.91 µs   7.24 µs   7.24 µs
Buffer.toString hex string                                     3.66 µs/iter     273,456.5      (3.6 µs … 4.02 µs)   3.64 µs   4.02 µs   4.02 µs
```

### String length 2^20
With massive lengths we the difference in ASCII and UTF-16 parsing
performance is enormous.

**This PR:**
```
benchmark                                                           time (avg)        iter/s             (min … max)       p75       p99      p995
-------------------------------------------------------------------------------------------------------------------- -----------------------------
Buffer.from ascii string                                              4.1 ms/iter         243.7     (2.64 ms … 6.74 ms)   4.43 ms   6.26 ms   6.74 ms
Buffer.from base64 string                                            3.74 ms/iter         267.6     (2.91 ms … 4.92 ms)   3.96 ms   4.31 ms   4.92 ms
Buffer.from utf16 string                                             7.72 ms/iter         129.5    (5.91 ms … 11.03 ms)   7.97 ms  11.03 ms  11.03 ms
Buffer.from hex string                                              35.72 ms/iter          28.0   (34.71 ms … 38.42 ms)  35.93 ms  38.42 ms  38.42 ms
Buffer.toString ascii string                                        78.92 ms/iter          12.7   (42.72 ms … 94.13 ms)  91.64 ms  94.13 ms  94.13 ms
Buffer.toString base64 string                                      833.62 µs/iter       1,199.6   (638.05 µs … 5.97 ms) 826.86 µs   2.45 ms   2.48 ms
Buffer.toString utf16 string                                        79.35 ms/iter          12.6    (69.72 ms … 88.9 ms)  86.66 ms   88.9 ms   88.9 ms
Buffer.toString hex string                                          31.04 ms/iter          32.2      (4.3 ms … 46.9 ms)  37.21 ms   46.9 ms   46.9 ms
```

**Main:**
```
benchmark                                                           time (avg)        iter/s             (min … max)       p75       p99      p995
-------------------------------------------------------------------------------------------------------------------- -----------------------------
Buffer.from ascii string                                            18.66 ms/iter          53.6   (15.61 ms … 23.26 ms)  20.62 ms  23.26 ms  23.26 ms
Buffer.from base64 string                                             4.7 ms/iter         212.9     (2.94 ms … 9.07 ms)   4.65 ms   9.06 ms   9.07 ms
Buffer.from utf16 string                                            33.49 ms/iter          29.9   (31.24 ms … 35.67 ms)  34.08 ms  35.67 ms  35.67 ms
Buffer.from hex string                                              39.38 ms/iter          25.4   (38.66 ms … 42.36 ms)  39.58 ms  42.36 ms  42.36 ms
Buffer.toString ascii string                                        77.68 ms/iter          12.9   (67.46 ms … 95.68 ms)  84.71 ms  95.68 ms  95.68 ms
Buffer.toString base64 string                                      825.53 µs/iter       1,211.3   (655.38 µs … 6.69 ms) 816.62 µs   3.07 ms   3.13 ms
Buffer.toString utf16 string                                        76.54 ms/iter          13.1    (66.9 ms … 85.26 ms)  83.63 ms  85.26 ms  85.26 ms
Buffer.toString hex string                                          38.56 ms/iter          25.9   (33.83 ms … 46.56 ms)  45.33 ms  46.56 ms  46.56 ms
```
2023-09-07 14:41:16 -06:00
Matt Mastracci
29784df24e
chore(ext/fs): port some ops to op2 (#20402)
Port as many of these ops as we can to `op2`. Waiting on a few
`deno_core` updates to complete this file.
2023-09-07 13:19:20 -06:00
Matt Mastracci
8eda8c0dbd
chore: make downloadPrebuilt a little more reliable (#20401)
Makes the prebuilt installation a bit more reliable:

 - Check for 200
 - Check for an executable header (MZ, ELF, etc)
 - Download to a .temp file until we're certain the file is valid
 - If multiple requests for a tool are made, only run one task
2023-09-07 17:13:11 +00:00
Matt Mastracci
9226207c01
chore(ext/node): port some ops to op2 (#20400) 2023-09-07 10:56:02 -06:00
David Sherret
3fc19dab47
feat: support import attributes (#20342) 2023-09-07 09:09:16 -04:00
Heyang Zhou
01a761f1d4
chore(ext/kv): limit total key size in an atomic op to 80 KiB (#20395)
Keys are expensive metadata. We track it for various purposes, e.g.
transaction conflict check, and key expiration.

This patch limits the total key size in an atomic operation to 80 KiB
(81920 bytes). This helps ensure efficiency in implementations.
2023-09-07 15:07:04 +08:00
Nayeem Rahman
2cbd1b40cb
fix(test): apply filter before checking for "only" (#20389) 2023-09-06 17:07:37 +01:00
Marvin Hagemeister
e0a269c23a
fix: don't show filtered test suites as running (#20385) 2023-09-06 14:54:21 +02:00
Bartek Iwańczuk
147c845c95
feat(test): Add Deno.test.ignore and Deno.test.only (#20365)
Closes https://github.com/denoland/deno/issues/17106
2023-09-06 14:17:33 +02:00
Divy Srivastava
9befa566ec
fix(ext/node): implement AES GCM cipher (#20368)
Adds support for AES-GCM 128/256 bit keys in `node:crypto` and
`setAAD()`, `setAuthTag()` and `getAuthTag()`

Uses https://github.com/littledivy/aead-gcm-stream

Fixes https://github.com/denoland/deno/issues/19836
https://github.com/denoland/deno/issues/20353
2023-09-06 11:01:50 +05:30
David Sherret
a0af53fea1
chore: pin third_party (#20386) 2023-09-06 06:07:55 +02:00
sigmaSd
be1fc754a1
feat(lsp): provide the deno.cache command server-side (#20111)
Co-authored-by: Nayeem Rahman <nayeemrmn99@gmail.com>
2023-09-05 16:36:35 +01:00
zuisong
4a561f12db
fix(node/child_process): don't crash on undefined/null value of an env var (#20378)
Fixes #20373

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-09-05 12:42:35 +02:00
Bartek Iwańczuk
2a1ba2732e
chore: disable 'test-http-content-length.js` test (#20344) 2023-09-04 23:23:01 +02:00
Bartek Iwańczuk
9e243d22f4
Revert "refactor: rewrite ops that use 'deferred' to use 'op2(async(lazy))' (#20303) (#20370)
This reverts commit
83426be6ee.

Includes a regression test.
2023-09-04 17:05:06 -04:00
Fabian
2cc7c8432f
fix(node): Bump hardcoded version to latest (#20366)
When trying to run
```
deno run -A --unstable npm:astro dev
```
in my Astro project it fails with:
```
Node.js v18.12.1 is not supported by Astro!
Please upgrade Node.js to a supported version: ">=18.14.1"
```
My current version is:
```
~ ❯ node --version
v20.5.1
```

Bumping the version to the latest stable Release of node in
`ext/node/polyfills/_process/process.ts` fixes this.
I don't know if this causes any conflicts, so please feel free to
correct me here.
2023-09-04 12:33:15 +02:00
Igor Zinkovsky
1dc5d42114
fix(ext/kv): add a warning for listenQueue if used with remote KV (#20341) 2023-09-03 17:47:52 -07:00
Divy Srivastava
d6c49353c3
chore(release): fix workspace formatter (#20357) 2023-09-03 10:09:15 +05:30
Nayeem Rahman
12f6ad32c2
fix(lsp): properly handle disabled configuration requests (#20358)
Fixes #19802.

Properly respect when clients do not have the `workspace/configuration`
capability, a.k.a. when an editor cannot provide scoped settings on
request from the LSP.

- Fix one spot where we weren't checking for the capability before
sending this request.
- For `enablePaths`, fall back to the settings passed in the
initialization options in more cases.
- Respect the `workspace/configuration` capability in the test harness
client.

See:
https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_configuration.
2023-09-02 16:36:04 +01:00
Bartek Iwańczuk
83426be6ee
refactor: rewrite ops that use 'deferred' to use 'op2(async(lazy))' (#20303)
Rewrites 3 ops that used "op(deferred)" to use "op2(async(lazy))"
instead.
This will allow us to remove codepath for handling "deferred" ops in
"deno_core".
2023-09-02 08:48:21 +02:00
Nayeem Rahman
e1fb48524d
Reland "feat(lsp): enable via config file detection (#20334)" (#20349) 2023-09-01 21:13:13 +01:00
Bartek Iwańczuk
2b191c6e9d
chore: forward v1.36.4 to main (#20352)
Co-authored-by: denobot <33910674+denobot@users.noreply.github.com>
2023-09-01 18:08:58 +00:00
Bartek Iwańczuk
c9223bc6a5
fix(evt/kv): Add serde feature to uuid (#20350)
`cargo publish` for v1.36.4 failed due to missing `serde` feature on
`uuid` dependency.
2023-09-01 18:41:35 +02:00
Bartek Iwańczuk
a74554987a
Revert "feat(lsp): enable via config file detection (#20334)" (#20347)
This reverts commit c0dcf6a357.

CC @nayeemrmn
2023-09-01 12:01:20 +00:00
Jakub Jirutka
3436f65e20
fix(ext/node): remove unnecessary and incorrect type priority_t (#20276)
`getpriority` and `setpriority` on musl libc accepts `int` / `c_int` /
`i32` as the first argument, not `u32`.

Since the `PRIO_PROCESS` constant is imported from the same crate (libc)
as the `getpriority` and `setpriority` functions, this type cast seems
to be completely unnecessary here.

It was introduced in aa8078b688 by
@crowlKats.

Relevant sources:

-
835661543d/src/unix/linux_like/linux/musl/mod.rs (L739-L740)
- https://git.musl-libc.org/cgit/musl/tree/src/misc/setpriority.c
- https://git.musl-libc.org/cgit/musl/tree/src/misc/getpriority.c

Co-authored-by: Aapo Alasuutari <aapo.alasuutari@gmail.com>
2023-09-01 11:21:23 +02:00
Bartek Iwańczuk
cd0fcc2257
chore: upgrade deno_core to 0.208.0 (#20340) 2023-08-31 19:56:25 +00:00
Nayeem Rahman
c0dcf6a357
feat(lsp): enable via config file detection (#20334)
With https://github.com/denoland/vscode_deno/pull/902 for
https://github.com/denoland/vscode_deno/issues/880.

For multi-folder workspaces, note that this only scans the first one and
applies the result to all. That means users would have to still have to
specify `"deno.enable": true/false` for their secondary folders if the
preference is different for those.
2023-08-31 19:10:09 +01:00
Filip Skokan
9d58c896dc
fix(ext/crypto): remove EdDSA alg key checks and export (#20331)
As per https://github.com/WICG/webcrypto-secure-curves/pull/24 this
removes the check for Ed25519 JWK `alg` during importKey and removes the
`alg` for Ed25519 keys during JWK exportKey.
2023-08-31 18:26:26 +05:30
Matt Mastracci
653e668c20
chore: bump deno_core to 0.207.0 (#20333)
rusty_v8 -> "0.75.1"
2023-08-30 21:45:47 +02:00
Nayeem Rahman
1cce306022
fix(runtime/permissions): Resolve executable specifiers in allowlists and queries (#14130)
Closes #14122.

Adds two extensions to `--allow-run` behaviour:
- When `--allow-run=foo` is specified and `foo` is found in the `PATH`
at startup, `RunDescriptor::Path(which("foo"))` is added to the
allowlist alongside `RunDescriptor::Name("foo")`. Currently only the
latter is.
- When run permission for `foo` is queried and `foo` is found in the
`PATH` at runtime, either `RunDescriptor::Path(which("foo"))` or
`RunDescriptor::Name("foo")` would qualify in the allowlist. Currently
only the latter does.
2023-08-30 18:52:01 +01:00
Nayeem Rahman
d28384c3de
refactor(lsp): store test definitions in adjacency list (#20330)
Previously:
```rust
pub struct TestDefinition {
  pub id: String,
  pub name: String,
  pub range: SourceRange,
  pub steps: Vec<TestDefinition>,
}

pub struct TestDefinitions {
  pub discovered: Vec<TestDefinition>,
  pub injected: Vec<lsp_custom::TestData>,
  pub script_version: String,
}
```
Now:
```rust
pub struct TestDefinition {
  pub id: String,
  pub name: String,
  pub range: Option<Range>,
  pub is_dynamic: bool, // True for 'injected' module, not statically detected but added at runtime.
  pub parent_id: Option<String>,
  pub step_ids: HashSet<String>,
}

pub struct TestModule {
  pub specifier: ModuleSpecifier,
  pub script_version: String,
  pub defs: HashMap<String, TestDefinition>,
}
```

Storing the test tree as a literal tree diminishes the value of IDs,
even though vscode stores them that way. This makes all data easily
accessible from `TestModule`. It unifies the interface between
'discovered' and 'injected' tests. This unblocks some enhancements wrt
syncing tests between the LSP and extension, such as this TODO:
61f08d5a71/client/src/testing.ts (L251-L259)
and https://github.com/denoland/vscode_deno/issues/900. We should also
get more flexibility overall.

`TestCollector` is cleaned up, now stores a `&mut TestModule` directly
and registers tests as it comes across them with
`TestModule::register()`. This method ensures sanity in the redundant
data from having both of `TestDefinition::{parent_id,step_ids}`.

All of the messy conversions between `TestDescription`,
`LspTestDescription`, `TestDefinition`, `TestData` and `TestIdentifier`
are cleaned up. They shouldn't have been using `impl From` and now the
full list of tests is available to their implementations.
2023-08-30 16:31:31 +01:00
Divy Srivastava
329698cf73
fix(napi): ignore tsfn recv error (#20324)
Ref https://github.com/denoland/deno/issues/20072
2023-08-30 09:04:55 +05:30
Ryan Dahl
f76bfbbe33
refactor(init): simplify template (#20325)
- Don't include benchmark file - most people won't need this.
- Use deno.json instead of deno.jsonc, because it's a more common file
  format.
2023-08-29 19:58:56 +00:00
Matt Mastracci
64045ebc7a
chore(cli): use NPM_CONFIG_REGISTRY for all tests (#20320)
We never want tests to hit the real npm registry because this causes
test flakes. In addition, we set a sentinal "unset" value for
`NPM_CONFIG_REGISTRY` to ensure that all tests requiring npm go through
the test server.
2023-08-29 13:02:54 -06:00
Igor Zinkovsky
441b860978
fix(ext/kv): don't panic if listening on queues and KV is not closed (#20317)
fixes #20312
2023-08-29 11:24:44 -07:00
David Sherret
c4451d3076
feat(lockfile): add redirects to the lockfile (#20262) 2023-08-29 12:03:02 -05:00
David Sherret
bdc91211b0
fix(lsp): recreate npm search cache when cache path changes (#20327) 2023-08-29 16:24:19 +00:00
Nayeem Rahman
b5f032df73
feat(lsp): npm specifier completions (#20121) 2023-08-29 10:22:05 -05:00
Bartek Iwańczuk
2929313652
fix(node/http): don't leak resources on destroyed request (#20040)
Closes https://github.com/denoland/deno/issues/19828
2023-08-29 12:13:58 +00:00
Nayeem Rahman
c31c93ce70
fix(lsp): delete test modules with all tests deleted (#20321)
Fixes https://github.com/denoland/vscode_deno/issues/899.
2023-08-29 09:43:11 +01:00
Yoshiya Hinosawa
fb7092fb43
fix(ext/node): fix argv[1] in Worker (#20305) 2023-08-29 12:18:25 +09:00
Nayeem Rahman
bd034e360d
refactor(runtime): factor out code between build.rs and worker.rs (#20299)
Adds `runtime/shared.rs` which is imported by both `build.rs` and the
rest of the crate, containing utilities used by both.
Renames the `snapshot_from_snapshot` feature to
`exclude_runtime_main_js` since that's what it does and it's relevant
outside of snapshotting when `__runtime_js_sources` is specified.
2023-08-28 15:30:46 -06:00
Matt Mastracci
7adaf613bf
fix(ext/node): shared global buffer unlock correctness fix (#20314)
The fix for #20188 was not entirely correct -- we were unlocking the
global buffer incorrectly. This PR introduces a lock state that ensures
we only unlock a lock we have taken out.
2023-08-28 15:28:39 -06:00
Matt Mastracci
9198bbd454
fix(ext/http): don't panic on stream responses in cancelled requests (#20316)
When a TCP connection is force-closed (ie: browser refresh), the
underlying future we pass to Hyper is dropped which may cause us to try
to drop the body resource while the OpState lock is still held.

Preconditions for this bug to trigger:

 - The body resource must have been taken
- The response must return a resource (which requires us to take the
OpState lock)
 - The TCP connection must have been dropped before this

Fixes #20315 and #20298
2023-08-28 13:29:34 -06:00
Marvin Hagemeister
539e5032d3
fix: unexpected lsp function arg comma completion (#20311) 2023-08-28 17:07:22 +02:00
David Sherret
ebfead1021
fix(fmt/markdown): ignore trailing words in code block info string for language detection (#20310) 2023-08-28 12:28:06 +00:00
osddeitf
c2547ba039
fix(node/http): correctly send Content-length header instead of Transfer-Encoding: chunked (#20127)
Fix #20063.
2023-08-28 09:32:54 +02:00
Jonathan Rezende
d22a6663fa
fix(network): adjust Listener type params (#18642)
Fixes https://github.com/denoland/deno/issues/18635
2023-08-27 20:55:04 +00:00
Nayeem Rahman
f826574873
test(bench): mark explicit timer test as flaky (#20304)
https://github.com/denoland/deno/actions/runs/5990043261/job/16246927846?pr=18642

The test is just `b.start(); b.end()` where the time measured between
must be less 10 μs during warmup. Can't improve it.
Also fixes doc mistake.
2023-08-27 22:22:23 +02:00
sigmaSd
916ddcef6d
feat(lint): --rules print all rules (#20256)
The motivation is If I'm using deno lint --rules, I want to see all the
rules especially the one that have no tags, since the recommend ones are
already active

This change also prints the tags associated with the rule inline.
2023-08-27 11:17:41 +02:00