This commit removes some of the technical debt related
to snapshotting JS code:
- "cli/ops/mod.rs" and "cli/build.rs" no longer define "cli" extension
which was not required anymore
- Cargo features for "deno_runtime" crate have been unified in
"cli/Cargo.toml"
- "cli/build.rs" uses "deno_runtime::snapshot::create_runtime_snapshot"
API
instead of copy-pasting the code
- "cli/js/99_main.js" was completely removed as it's not necessary
anymore
Towards https://github.com/denoland/deno/issues/21137
This makes `CliNpmResolver` a trait. The terminology used is:
- **managed** - Deno manages the node_modules folder and does an
auto-install (ex. `ManagedCliNpmResolver`)
- **byonm** - "Bring your own node_modules" (ex. `ByonmCliNpmResolver`,
which is in this PR, but unimplemented at the moment)
Part of #18967
Closes #19399 (running without snapshots at all was suggested as an
alternative solution).
Adds a `__runtime_js_sources` pseudo-private feature to load extension
JS sources at runtime for faster development, instead of building and
loading snapshots or embedding sources in the binary. Will only work in
a development environment obviously.
Try running `cargo test --features __runtime_js_sources
integration::node_unit_tests::os_test`. Then break some behaviour in
`ext/node/polyfills/os.ts` e.g. make `function cpus() {}` return an
empty array, and run it again. Fix and then run again. No more build
time in between.
…nclusion" (#19519)"
This reverts commit 28a4f3d0f5.
This change causes failures when used outside Deno repo:
```
============================================================
Deno has panicked. This is a bug in Deno. Please report this
at https://github.com/denoland/deno/issues/new.
If you can reliably reproduce this panic, include the
reproduction steps and re-run with the RUST_BACKTRACE=1 env
var set and include the backtrace in your report.
Platform: linux x86_64
Version: 1.34.3+b37b286
Args: ["/opt/hostedtoolcache/deno/0.0.0-b37b286f7fa68d5656f7c180f6127bdc38cf2cf5/x64/deno", "test", "--doc", "--unstable", "--allow-all", "--coverage=./cov"]
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Failed to read "/home/runner/work/deno/deno/core/00_primordials.js"
Caused by:
No such file or directory (os error 2)', core/runtime/jsruntime.rs:699:8
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
Relands #19463. This time the `ExtensionFileSourceCode` enum is
preserved, so this effectively just splits feature
`include_js_for_snapshotting` into `exclude_js_sources` and
`runtime_js_sources`, adds a `force_include_js_sources` option on
`extension!()`, and unifies `ext::Init_ops_and_esm()` and
`ext::init_ops()` into `ext::init()`.
… (#19463)"
This reverts commit ceb03cfb03.
This is being reverted because it causes 3.5Mb increase in the binary
size,
due to runtime JS code being included in the binary, even though it's
already snapshotted.
CC @nayeemrmn
Remove `ExtensionFileSourceCode::LoadedFromFsDuringSnapshot` and feature
`include_js_for_snapshotting` since they leak paths that are only
applicable in this repo to embedders. Replace with feature
`exclude_js_sources`. Additionally the feature
`force_include_js_sources` allows negating it, if both features are set.
We need both of these because features are additive and there must be a
way of force including sources for snapshot creation while still having
the `exclude_js_sources` feature. `force_include_js_sources` is only set
for build deps, so sources are still excluded from the final binary.
You can also specify `force_include_js_sources` on any extension to
override the above features for that extension. Towards #19398.
But there was still the snapshot-from-snapshot situation where code
could be executed twice, I addressed that by making `mod_evaluate()` and
scripts like `core/01_core.js` behave idempotently. This allowed
unifying `ext::init_ops()` and `ext::init_ops_and_esm()` into
`ext::init()`.
This is just a straight refactor and I didn't do any cleanup in
ext/node. After this PR we can start to clean it up and make things
private that don't need to be public anymore.
This commit changes the build process in a way that preserves already
registered ops in the snapshot. This allows us to skip creating hundreds of
"v8::String" on each startup, but sadly there is still some op registration
going on startup (however we're registering 49 ops instead of >200 ops).
This situation could be further improved, by moving some of the ops
from "runtime/" to a separate extension crates.
---------
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Follow-up to #18210:
* we are passing the generated `cfg` object into the state function
rather than passing individual config fields
* reduce cloning dramatically by making the state_fn `FnOnce`
* `take` for `ExtensionBuilder` to avoid more unnecessary copies
* renamed `config` to `options`
This implements two macros to simplify extension registration and centralize a lot of the boilerplate as a base for future improvements:
* `deno_core::ops!` registers a block of `#[op]`s, optionally with type
parameters, useful for places where we share lists of ops
* `deno_core::extension!` is used to register an extension, and creates
two methods that can be used at runtime/snapshot generation time:
`init_ops` and `init_ops_and_esm`.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
There's no point for this API to expect result. If something fails it should
result in a panic during build time to signal to embedder that setup is
wrong.
This PR adds copies of several unstable APIs that are available
in "Deno[Deno.internal].nodeUnstable" namespace.
These copies do not perform unstable check (ie. don't require
"--unstable" flag to be present). Otherwise they work exactly
the same, including permission checks.
These APIs are not meant to be used by users directly and
can change at any time.
Copies of following APIs are available in that namespace:
- Deno.spawnChild
- Deno.spawn
- Deno.spawnSync
- Deno.serve
- Deno.upgradeHttpRaw
- Deno.listenDatagram
This commit adds "deno bench" subcommand and "Deno.bench()"
API that allows to register bench cases.
The API is modelled after "Deno.test()" and "deno test" subcommand.
Currently the output is rudimentary and bench cases and not
subject to "ops" and "resource" sanitizers.
Co-authored-by: evan <github@evan.lol>
This commit adds support for running test in parallel.
Entire test runner functionality has been rewritten
from JavaScript to Rust and a set of ops was added to support reporting in Rust.
A new "--jobs" flag was added to "deno test" that allows to configure
how many threads will be used. When given no value it defaults to 2.
This commits adds adds "permissions" option to the test definitions
which allows tests to run with different permission sets than
the process's permission.
The change will only be in effect within the test function, once the
test has completed the original process permission set is restored.
Test permissions cannot exceed the process's permission.
You can only narrow or drop permissions, failure to acquire a
permission results in an error being thrown and the test case will fail.
This commit moves Deno JS runtime, ops, permissions and
inspector implementation to new "deno_runtime" crate located
in "runtime/" directory.
Details in "runtime/README.md".
Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
This commit does major refactor of "Worker" and "WebWorker",
in order to decouple them from "ProgramState" and "Flags".
The main points of interest are "create_main_worker()" and
"create_web_worker_callback()" functions which are responsible
for creating "Worker" and "WebWorker" in CLI context.
As a result it is now possible to factor out common "runtime"
functionality into a separate crate.
This ports the REPL over to Rust and makes use of an inspector session to run a REPL on top of any isolate which lets make full use of rustylines various things like validators and completors without having to introduce a bunch of hard to test internal ops and glue code.
An accidental but good side effect of this is that the multiple line input we previously had is now an editable multi-line input prompt that is correctly stored in the history as a single entry.
This commits removes "CompilerWorker" in favor of
using "JsRuntime".
"cli/ops/compiler.rs" has been removed in favor of inline
registration of ops in "cli/tsc.rs"
- remove "CliState.workers" and "CliState.next_worker_id", instead
store them on "OpState" using type aliases.
- remove "CliState.global_timer" and "CliState.start_time", instead
store them on "OpState" using type aliases.
- remove "CliState.is_internal", instead pass it to Worker::new
- move "CliState::permissions" to "OpState"
- move "CliState::main_module" to "OpState"
- move "CliState::global_state" to "OpState"
- move "CliState::check_unstable()" to "GlobalState"
- change "cli_state()" to "global_state()"
- change "deno_core::ModuleLoader" trait to pass "OpState" to callbacks
- rename "CliState" to "CliModuleLoader"
Moves op_close and op_resources to deno_core::ops and exports them.
Adds serde dependency to deno_core and reexports it.
Moves JS implementation of those ops to Deno.core and reexports them in Deno.
Provides a concrete state type that can be dynamically added. This is necessary for op crates.
* renames BasicState to OpState
* async ops take `Rc<RefCell<OpState>>`
* sync ops take `&mut OpState`
* removes `OpRegistry`, `OpRouter` traits
* `get_error_class_fn` moved to OpState
* ResourceTable moved to OpState
