The same issue in two different places - doing blocking FS work in an
async task, limiting the amount of work that happens concurrently.
- When setting up node_modules, where we try to set up entries
concurrently but were blocking other tasks from actually running.
- When loading package info from the npm registry file cache, loading
and deserializing is expensive and prevents concurrency. This was
especially noticeable when loading an npm resolution snapshot from a
lockfile (`snapshot_from_lockfile` in `deno_npm`).
Installing deps in `deno-docs`:
```
❯ hyperfine -i -p 'rm -rf node_modules/' '../d7/deno-main i' '../d7/target/release/deno i'
Benchmark 1: ../d7/deno-main i
Time (mean ± σ): 2.193 s ± 0.027 s [User: 0.589 s, System: 1.033 s]
Range (min … max): 2.151 s … 2.242 s 10 runs
Benchmark 2: ../d7/target/release/deno i
Time (mean ± σ): 1.597 s ± 0.021 s [User: 0.977 s, System: 1.337 s]
Range (min … max): 1.550 s … 1.627 s 10 runs
Summary
../d7/target/release/deno i ran
1.37 ± 0.02 times faster than ../d7/deno-main i
```
Caching `npm:@11ty/eleventy`:
```
❯ hyperfine -i -p 'rm -rf node_modules/' --warmup 5 '../../d7/deno-main cache npm:@11ty/eleventy' '../../d7/target/release/deno cache npm:@11ty/eleventy'
Benchmark 1: ../../d7/deno-main cache npm:@11ty/eleventy
Time (mean ± σ): 129.9 ms ± 2.2 ms [User: 27.5 ms, System: 101.3 ms]
Range (min … max): 127.5 ms … 135.8 ms 10 runs
Benchmark 2: ../../d7/target/release/deno cache npm:@11ty/eleventy
Time (mean ± σ): 100.6 ms ± 1.3 ms [User: 38.8 ms, System: 233.8 ms]
Range (min … max): 99.3 ms … 103.2 ms 10 runs
Summary
../../d7/target/release/deno cache npm:@11ty/eleventy ran
1.29 ± 0.03 times faster than ../../d7/deno-main cache npm:@11ty/eleventy
```
---------
Co-authored-by: David Sherret <dsherret@gmail.com>
This commit changes `gzip` compression in `Deno.serve` API to flush data
after each write. There's a slight performance regression, but provided
test shows a scenario that was not possible before.
---------
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Hard linking (`linkat`) is ridiculously slow on mac. `copyfile` is
better, but what's even faster is `clonefile`. It doesn't have the space
savings that comes with hardlinking, but the performance difference is
worth it imo.
```
❯ hyperfine -i -p 'rm -rf node_modules/' '../../d7/target/release/deno cache npm:@11ty/eleventy' 'deno cache npm:@11ty/eleventy'
Benchmark 1: ../../d7/target/release/deno cache npm:@11ty/eleventy
Time (mean ± σ): 115.4 ms ± 1.2 ms [User: 27.2 ms, System: 87.3 ms]
Range (min … max): 113.7 ms … 117.5 ms 10 runs
Benchmark 2: deno cache npm:@11ty/eleventy
Time (mean ± σ): 619.3 ms ± 6.4 ms [User: 34.3 ms, System: 575.6 ms]
Range (min … max): 612.2 ms … 633.3 ms 10 runs
Summary
../../d7/target/release/deno cache npm:@11ty/eleventy ran
5.37 ± 0.08 times faster than deno cache npm:@11ty/eleventy
```
# Summary
This PR resolves about the issue.
fixes #10810
And the formerly context is in the PR.
#22582
Here is an expected behaviour example with this change.
- 🦕.test.ts
```ts
import { assertEquals } from "https://deno.land/std@0.215.0/assert/mod.ts";
Deno.test("example test", () => {
assertEquals("🍋", "🦕");
});
```
We didn't honour the `position` options of `fd.read` and `fd.write`
because we checked if the buffer is of type `Buffer` instead of just
`Uint8Array`. Node does the latter. In doing so I noticed that the file
handle id was written to a public property which it definitely shouldn't
be. This was probably a typo.
Fixes https://github.com/denoland/deno/issues/23707
We use the `target` property of the snapshot options to derive
`process.platform` and `process.arch` from. This value had an incorrect
format when compiled with `__runtime_js_sources` enabled. This PR fixes
that so that `process.platform` holds the proper value.
Fixes https://github.com/denoland/deno/issues/23164
We didn't support the `throwIfNoEntry` option for Node's `fs.lstatSync`
method. Note that the async variant doesn't have this option.
Fixes https://github.com/denoland/deno/issues/23996
The mixed `number | bigint` representation was useful optimization for
pointers. Now, pointers are represented as V8 externals. As part of the
FFI stabilization effort we want to make `bigint` the only
representation for `u64` and `i64`.
BigInt representation performance is almost on par with mixed
representation with the added benefit that its less confusing and users
don't need manual checks and conversions for doing operations on the
value.
```
cpu: AMD Ryzen 5 7530U with Radeon Graphics
runtime: deno 1.43.6+92a8d09 (x86_64-unknown-linux-gnu)
file:///home/divy/gh/ffi/main.ts
benchmark time (avg) iter/s (min … max) p75 p99 p995
-------------------------------------------------------------------------- -----------------------------
nop 4.01 ns/iter 249,533,690.5 (3.97 ns … 10.8 ns) 3.97 ns 4.36 ns 9.03 ns
ret bigint 7.74 ns/iter 129,127,186.8 (7.72 ns … 10.46 ns) 7.72 ns 8.11 ns 8.82 ns
ret i32 7.81 ns/iter 128,087,100.5 (7.77 ns … 12.72 ns) 7.78 ns 8.57 ns 9.75 ns
ret bigint (add op) 15.02 ns/iter 66,588,253.2 (14.64 ns … 24.99 ns) 14.76 ns 19.13 ns 19.44 ns
ret i32 (add op) 12.02 ns/iter 83,209,131.8 (11.95 ns … 18.18 ns) 11.98 ns 13.11 ns 14.5 ns
```
Closes #19697. This fixes a bug where the writeFile API can create
partially-overwritten files which may lead to invalid / corrupt files or
data leakage. It also aligns the behavior of writing a ReadableStream
and writing a Uint8Array to the disk.
`createAsyncFromSyncIterator(x)` which is used in
`ReadableStream.from()` expects `x` as `Iterable` but, previous
implements specify `Iterator` or `IterableIterator`. If it was
`IterableIterator`, it would work, but if it was `Iterator`, an
exception will occur.
Tests have been merged into WPT.
https://github.com/web-platform-tests/wpt/pull/46365
---------
Co-authored-by: Asher Gomez <ashersaupingomez@gmail.com>
Enhanced warning message for --env flag with run and eval subcommands.
The commit is specifically made to address issue #23674 by improving the
warning messages that appear when using the --env flag with run or eval
subcommands in the following scenarios:
1. Missing environment file.
2. Incorrect syntax in the environment file content.
**Changes made**
- Distinguishes between cases of missing environment file and wrong
syntax in the environment file content.
- Shows a concise warning message to convey the case/issue occurred.
**Code changes & enhancements**
- Implemented a match statement to handle different types of errors
received while getting and parsing the file content to display a concise
warning message, rather than simple error check and then displaying the
same warning message for whatever the type of error is.
- Updated the related existing tests to reflect the new warning
messages.
- Added two test cases to cover the wrong environment file content
syntax with both run and eval subcommands.
**Impact**
The use of --env flag with both run/eval would be more user-friendly as
it gives a precise description of what is not right when using
incorrectly.
If you could give it a look, @dsherret , I appreciate your feedback on
these changes.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Node sets the default HTTP response status code to 200 on the
`ServerResponse`. We initialised it as `undefined` before which caused a
problem with 11ty's dev server.
Thanks to @vrugtehagel for reporting this issue and finding the correct
fix as well 🎉
Fixes https://github.com/denoland/deno/issues/23970
This commit adds initial support for ".npmrc" files.
Currently we only discover ".npmrc" files next to "package.json" files
and discovering these files in user home dir is left for a follow up.
This pass supports "_authToken" and "_auth" configuration
for providing authentication.
LSP support has been left for a follow up PR.
Towards https://github.com/denoland/deno/issues/16105
Fixes #23571.
Previously, we required a `deno.json` to be present (or the `--lock`
flag) in order for us to resolve a `deno.lock` file. This meant that if
you were using deno in an npm-first project deno wouldn't use a
lockfile.
Additionally, while I was fixing that, I discovered there were a couple
bugs keeping the future `install` command from using a lockfile.
With this PR, `install` will actually resolve the lockfile (or create
one if not present), and update it if it's not up-to-date. This also
speeds up `deno install`, as we can use the lockfile to skip work during
npm resolution.
The flag lets us exit from read loop without throwing an error when
the stream is cancelled.
This fixes gRPC cancellation example.
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This PR removes the use of the custom `utc_now` function in favor of the
`chrono` implementation. It resolves #22864.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
If `@@asyncIterator` is `null` or `undefined`, it should ignores and
fallback to `@@iterator`.
Tests have been merged into WPT.
https://github.com/web-platform-tests/wpt/pull/46374
The proposal of `ReadableStream.from` uses TC39 [GetIterator][] and
[GetMethod][] within it.
GetMethod treats null as undefined.
So if `@@asyncIterator` is `null` it should be ignored and fallback to
`@@iterator`.
[GetIterator]: https://tc39.es/ecma262/#sec-getiterator
[GetMethod]: https://tc39.es/ecma262/#sec-getmethod
```bash
> deno eval "ReadableStream.from({ [Symbol.asyncIterator]: null, [Symbol.iterator]: () => ({ next: () => ({ done: true }) }) }).pipeTo(new WritableStream())"
error: Uncaught (in promise) TypeError: obj[SymbolAsyncIterator] is not a function
ReadableStream.from({ [Symbol.asyncIterator]: null, [Symbol.iterator]: () => ({ next: () => ({ done: true }) }) }).pipeTo(new WritableStream())
^
at getIterator (ext:deno_web/06_streams.js:5105:38)
at Function.from (ext:deno_web/06_streams.js:5207:22)
at file:///D:/work/js/deno/tests/wpt/suite/$deno$eval:1:16
```
---------
Co-authored-by: Asher Gomez <ashersaupingomez@gmail.com>
While investigating poor cold start performance on my GCP VM (32 cores,
130GB SSD), I found that writing to the various sqlite databases in
DENO_DIR was quite slow. The slowness seems to primarily be caused by
excessive latency from a number of `fsync()` calls.
The performance difference is best demonstrated by deleting the sqlite
databases from DENO_DIR while leaving the downloaded sources in place.
The benchmark (see notes below):
```
piscisaureus@bert-us:~/erofs/source$ export DENO_DIR=./.deno
piscisaureus@bert-us:~/erofs/source$ hyperfine --warmup 3 \
--prepare "rm -rf .deno/*_v1*" \
"deno run -A --cached-only demo.ts" \
"eatmydata deno run -A --cached-only demo.ts" \
"~/deno/target/release/deno run -A --cached-only demo.ts"
Benchmark 1: deno run -A --cached-only demo.ts
Time (mean ± σ): 1.174 s ± 0.037 s [User: 0.153 s, System: 0.184 s]
Range (min … max): 1.104 s … 1.212 s 10 runs
Benchmark 2: eatmydata deno run -A --cached-only demo.ts
Time (mean ± σ): 265.5 ms ± 3.6 ms [User: 138.5 ms, System: 135.1 ms]
Range (min … max): 260.6 ms … 271.2 ms 11 runs
Benchmark 3: ~/deno/target/release/deno run -A --cached-only demo.ts
Time (mean ± σ): 226.2 ms ± 9.2 ms [User: 136.7 ms, System: 93.3 ms]
Range (min … max): 218.8 ms … 247.1 ms 13 runs
Summary
~/deno/target/release/deno run -A --cached-only demo.ts ran
1.17 ± 0.05 times faster than eatmydata deno run -A --cached-only demo.ts
5.19 ± 0.27 times faster than deno run -A --cached-only demo.ts
```
Notes:
* Benchmark 1: unmodified Deno 1.43.6
* Benchmark 2: unmodified Deno 1.43.6 wrapped with `eatmydata` (which is
a tool to neuter `fsync()` calls)
* Benchmark 3: this PR applied on top of Deno 1.43.6
The script that got benchmarked:
```typescript
// demo.ts
import * as express from "npm:express@4.16.3";
import * as postgres from "https://deno.land/x/postgres/mod.ts";
let _dummy = [express, postgres]; // Force use of imports.
console.log("hello world");
```
This is a primordialization effort to improve resistance against users
tampering with the global `Object` prototype.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
