1. Rewrites the tests to be more back and forth rather than getting the
output all at once (which I believe was causing the hangs on linux and
maybe mac)
2. Runs the pty tests on the linux ci.
3. Fixes a bunch of tests that were just wrong.
4. Adds timeouts on the pty tests.
This commit changes how data required to bootstrap main and worker
runtime is serialized.
Instead of relying on serde_v8 and using JSON object,
we're doing manual serialization to a "v8::Array". This limits number
of V8 strings that need to be serialized by 16.
It also made it clear that some data could be obtained during
snapshotting instead of during bootstrap.
Since we can preserve ops in the snapshot these days, we no longer
need to have "Deno[Deno.internal].nodeUnstable" namespace.
Instead, various built-in Node.js modules can use appropriate APIs
directly.
This commit adds unstable "Deno.openKv()" API that allows to open
a key-value database at a specified path.
---------
Co-authored-by: Luca Casonato <hello@lcas.dev>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This PR _**temporarily**_ removes WebGPU (which has behind the
`--unstable` flag in Deno), due to performance complications due to its
presence.
It will be brought back in the future; as a point of reference, Chrome
will ship WebGPU to stable on 26/04/2023.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit changes how "unload" event is handled - before
this commit an event listener was added unconditionally in
the runtime bootstrapping function, which for some reason was
very expensive (0.3ms). Instead of adding an event listener,
a check was added to "dispatchEvent" function that performs
the same action (so it's only called if there's an event dispatched).
This commit renames "deno_core::InternalModuleLoader" to
"ExtModuleLoader" and changes the specifiers used by the
modules loaded from this loader to "ext:".
"internal:" scheme was really ambiguous and it's more characters than
"ext:", which should result in slightly smaller snapshot size.
Closes https://github.com/denoland/deno/issues/18020
This commit moves some of the code from "99_main.js" to
be executed during the snapshot time instead of on each
worker bootstrap. These should minimally help with startup
time benchmark.
This API is required by several extensions like "ext/node", "ext/ffi"
and also FS APIs that we want to move to a separate crate. Because
of that "pathFromURL" API was moved to "deno_web" extension so
other extension crates can rely on it.
We use information about build in several extension crates like
"ext/node" or "runtime/". In an effort to move "fs" APIs to a separate
crate it is a prerequisite to have this information available outside
of the "runtime/" crate.
This commit moves definition of "build" object to "Deno.core" that is
later forwarded to "Deno.build".
This commit merges "runtime/js/40_spawn.js" into
"runtime/js/40_process.js", and "runtime::ops::spawn"
into "runtime::ops::process".
It makes little sense to have them separated given that we want to
factor out these APIs into a separate extension crate.
JavaScript APIs from "runtime/js/40_files.js" combined abstractions
for stdio streams ("Stdout", "Stderr", "Stdin") and file system file
("File", "FsFile"). APIs from "runtime/js/40_read_file.js" and
"runtime/js/40_write_file.js" were implemented using ops from
"runtime/ops/fs.rs".
This file was removed and relevant APIs were moved to "deno_io/12_io.js"
and "runtime/js/30_fs.js".
This work is meant to enable factoring out "deno_fs" crate.
This file was not used anywhere - even though it exposed an unstable
"Deno.DiagnosticCategory" API. Seems like a left over from when we
had "Deno.emit" API.
This commit changes "include_js_files!" macro from "deno_core"
in a way that "dir" option doesn't cause specifiers to be rewritten
to include it.
Example:
```
include_js_files! {
dir "js",
"hello.js",
}
```
The above definition required embedders to use:
`import ... from "internal:<ext_name>/js/hello.js"`.
But with this change, the "js" directory in which the files are stored
is an implementation detail, which for embedders results in:
`import ... from "internal:<ext_name>/hello.js"`.
The directory the files are stored in, is an implementation detail and
in some cases might result in a significant size difference for the
snapshot. As an example, in "deno_node" extension, we store the
source code in "polyfills" directory; which resulted in each specifier
to look like "internal:deno_node/polyfills/<module_name>", but with
this change it's "internal:deno_node/<module_name>".
Given that "deno_node" has over 100 files, many of them having
several import specifiers to the same extension, this change removes
10 characters from each import specifier.
Runtime generation of async op wrappers contributed to increased startup
time and core became unusable with
`--disallow-code-generation-from-strings` flag. The optimization only
affects very small microbenchmarks so this revert will not cause any
regressions.
This commit further improves startup time by:
- no relying on "JsRuntime::execute_script" for runtime bootstrapping,
this is instead done using V8 APIs directly
- registering error classes during the snapshot time, instead of on
startup
Further improvements can be made, mainly around removing
"core.initializeAsyncOps()" which takes around 2ms.
This commit should result in ~1ms startup time improvement.
Adds two test files: "cli/tests/unit_node/process_test.ts" and
"cli/tests/unit_node/child_process_test.ts"
---------
Co-authored-by: Yoshiya Hinosawa <stibium121@gmail.com>
This commit moves "deno_std/node" in "ext/node" crate. The code is
transpiled and snapshotted during the build process.
During the first pass a minimal amount of work was done to create the
snapshot, a lot of code in "ext/node" depends on presence of "Deno"
global. This code will be gradually fixed in the follow up PRs to migrate
it to import relevant APIs from "internal:" modules.
Currently the code from snapshot is not used in any way, and all
Node/npm compatibility still uses code from
"https://deno.land/std/node" (or from the location specified by
"DENO_NODE_COMPAT_URL"). This will also be handled in a follow
up PRs.
---------
Co-authored-by: crowlkats <crowlkats@toaxl.com>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Co-authored-by: Yoshiya Hinosawa <stibium121@gmail.com>
This is a proof of concept for being able to snapshot TypeScript files.
Currently only a single runtime file is authored in TypeScript -
"runtime/js/01_version.ts".
Not needed infrastructure was removed from "core/snapshot_util.rs".
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This PR refactors all internal js files (except core) to be written as
ES modules.
`__bootstrap`has been mostly replaced with static imports in form in
`internal:[path to file from repo root]`.
To specify if files are ESM, an `esm` method has been added to
`Extension`, similar to the `js` method.
A new ModuleLoader called `InternalModuleLoader` has been added to
enable the loading of internal specifiers, which is used in all
situations except when a snapshot is only loaded, and not a new one is
created from it.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit adds sync versions of async APIs to "Deno.permissions"
namespace.
Following APIs were added:
- "Deno.permissions.querySync"
- "Deno.permissions.requestSync"
- "Deno.permissions.revokeSync"
This commit removes "Deno.core" namespace. It is strictly private API
that has no stability guarantees, we were supposed to remove it long time ago.
Co-authored-by: Yoshiya Hinosawa <stibium121@gmail.com>
Updated third_party dlint to v0.37.0 for GitHub Actions. This PR
includes following changes:
* fix(prefer-primordials): Stop using array pattern assignments
* fix(prefer-primordials): Stop using global intrinsics except for
`SharedArrayBuffer`
* feat(guard-for-in): Apply new guard-for-in rule
Previously, `Deno.permissions.[revoke|request]()` wouldn't correctly
process the `path: URL` when `name` was `ffi`. This change fixes that
behaviour and adds a new function, `formDescriptor()`, to ensure `URL`
arguments are consistently handled across
`Deno.permissions.[query|revoke|request]()`.
This commit changes implementation of "Deno.memoryUsage()" to return
correct value for "rss" field. To do that we implement a specialized function
per os to retrieve this information.
This commit removes three unstable Deno APIs:
- "Deno.spawn()"
- "Deno.spawnSync()"
- "Deno.spawnChild()"
These APIs were replaced by a unified "Deno.Command" API.
Refactors the `Deno.Command` class to not handle any state, but only being an intermediary to calling its methods, and as such any methods and properties besides `output`, `outputSync` & `spawn` have been removed. Interracting with a `spawn`ed subprocess now works by using the methods and properties on the returned class of the `spawn` method.
**This patch**
```
benchmark time (avg) (min … max) p75 p99 p995
------------------------------------------------- -----------------------------
echo deno 23.99 ms/iter (22.51 ms … 33.61 ms) 23.97 ms 33.61 ms 33.61 ms
cat 16kb 24.27 ms/iter (22.5 ms … 35.21 ms) 24.2 ms 35.21 ms 35.21 ms
cat 1mb 25.88 ms/iter (25.04 ms … 30.28 ms) 26.12 ms 30.28 ms 30.28 ms
cat 15mb 38.41 ms/iter (35.7 ms … 50 ms) 38.31 ms 50 ms 50 ms
```
**main**
```
benchmark time (avg) (min … max) p75 p99 p995
------------------------------------------------- -----------------------------
echo deno 35.66 ms/iter (34.53 ms … 41.84 ms) 35.79 ms 41.84 ms 41.84 ms
cat 16kb 35.99 ms/iter (34.52 ms … 44.94 ms) 36.05 ms 44.94 ms 44.94 ms
cat 1mb 38.68 ms/iter (36.67 ms … 50.44 ms) 37.95 ms 50.44 ms 50.44 ms
cat 15mb 48.4 ms/iter (46.19 ms … 58.41 ms) 49.16 ms 58.41 ms 58.41 ms
```
This PR adds copies of several unstable APIs that are available
in "Deno[Deno.internal].nodeUnstable" namespace.
These copies do not perform unstable check (ie. don't require
"--unstable" flag to be present). Otherwise they work exactly
the same, including permission checks.
These APIs are not meant to be used by users directly and
can change at any time.
Copies of following APIs are available in that namespace:
- Deno.spawnChild
- Deno.spawn
- Deno.spawnSync
- Deno.serve
- Deno.upgradeHttpRaw
- Deno.listenDatagram
V8's JIT can do a better job knowing the argument count and also enable
fast call path (in future).
This also lets us call async ops without `opAsync`:
```js
const { ops } = Deno.core;
await ops.op_void_async();
```
this patch: 4405286 ops/sec
main: 3508771 ops/sec
This commit stabilizes "Deno.consoleSize()" API.
There is one change compared to previous unstable API,
in that the API doesn't accept any arguments. Console size
is established by querying syscalls for stdio streams at fd
0, 1 and 2.
Previously `op_net_listen`, `op_net_accept`, and various other ops in
ext/net where variadic on the transport. This created a lot of code
bloat. This commit updates the code to instead have separate ops for
each transport.
This change adds `windowsRawArguments` to `SpawnOptions`. The option enables
skipping the default quoting and escaping while creating the command on
windows.
The option works in a similar way as `windowsVerbatimArguments` in
child_process.spawn options in Node.js, and is necessary for simulating
it in `std/node`.
closes #8852
This commit adds a new op_write_all to core that allows writing an
entire chunk in a single async op call. Internally this calls
`Resource::write_all`.
The `writableStreamForRid` has been moved to `06_streams.js` now, and
uses this new op. Various other code paths now also use this new op.
Closes #16227
We can use Resource::read_return & op_read instead. This allows HTTP
request bodies to participate in FastStream.
To make this work, `readableStreamForRid` required a change to allow non
auto-closing resources to be handled. This required some minor changes
in our FastStream paths in ext/http and ext/flash.
