When we were doing single process in-memory reports getting the source
from the runtime was practical, but now that we're writing to disk this
conflicts with the format tools taking raw v8 coverage dumps expect.
This commit adds new option to "Worker" Web API that allows to
configure permissions.
New "Worker.deno.permissions" option can be used to define limited
permissions to the worker thread by either:
- inherit set of parent thread permissions
- use limited subset of parent thread permissions
- revoke all permissions (full sandbox)
In order to achieve this functionality "CliModuleLoader"
was modified to accept "initial permissions", which are used
for top module loading (ie. uses parent thread permission set
to load top level module of a worker).
This commit removes "js" module from "cli".
It contained stuff related to TypeScript compiler (snapshot,
declaration files) and thus it was moved to "tsc" module.
This PR refactors "cli/flags.rs" and "runtime/permissions.rs" so
that "allow_read", "allow_write" and "allow_net" themselves
have allowlists, instead of storing them in additional fields.
This commit moves Deno JS runtime, ops, permissions and
inspector implementation to new "deno_runtime" crate located
in "runtime/" directory.
Details in "runtime/README.md".
Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
This commit does major refactor of "Worker" and "WebWorker",
in order to decouple them from "ProgramState" and "Flags".
The main points of interest are "create_main_worker()" and
"create_web_worker_callback()" functions which are responsible
for creating "Worker" and "WebWorker" in CLI context.
As a result it is now possible to factor out common "runtime"
functionality into a separate crate.
This commit makes the file watcher continue to work even if module
resolution fails at the initial attempt, allowing us to execute `run`
or `bundle` subcommand when a script has invalid syntax. In such
cases, the watcher observes a single file that is specified as an
command line argument.
Factored out "init_v8_flags", "init_logger" and
"get_subcommand" from "main" function.
Also "Worker" was removed in favor of moving
logic to "MainWorker" and "WebWorker" respectively.
This commit adds support for "--watch" flag for "bundle"
and "fmt" subcommands.
In addition to this, it refactors "run --watch" command so that
module resolution will occur every time the file watcher detects
file addition/deletion, which allows the watcher to observe a file
that is newly added to the dependency as well.
This commit moves following tools into a single "tools"
module located at "cli/tools/mod.rs":
- formatter
- linter
- test runner
- coverage collector
- installer
- binary upgrader
- repl
This commit renames "fs" module in "cli/" to "fs_util". This is purely
cosmetic change; there were a few places which aliased "crate::fs"
to "deno_fs" which was very confusing with "fs" module in ops.
This commit removes ProgramState::permissions field.
Having permissions parsed from CLI flags stored on globally
accessible state object made it easy to mistakenly use these
permissions in situations which required "runtime" permissions.
* Revert "refactor: Worker is not a Future (#7895)"
This reverts commit f4357f0ff9.
* Revert "refactor(core): JsRuntime is not a Future (#7855)"
This reverts commit d8879feb8c.
* Revert "fix(core): module execution with top level await (#7672)"
This reverts commit c7c7677825.
This commit rewrites deno::Worker to not implement Future
trait.
Instead there are two separate methods:
- Worker::poll_event_loop() - does single tick of event loop
- Worker::run_event_loop() - runs event loop to completion
Additionally some cleanup to Worker's field visibility was done.
This ports the REPL over to Rust and makes use of an inspector session to run a REPL on top of any isolate which lets make full use of rustylines various things like validators and completors without having to introduce a bunch of hard to test internal ops and glue code.
An accidental but good side effect of this is that the multiple line input we previously had is now an editable multi-line input prompt that is correctly stored in the history as a single entry.
This removes the dependency on global state and instead relies on the
runtime's internal state to get the script sources it saw when it
collected code coverage for them.
- remove "CliState.workers" and "CliState.next_worker_id", instead
store them on "OpState" using type aliases.
- remove "CliState.global_timer" and "CliState.start_time", instead
store them on "OpState" using type aliases.
- remove "CliState.is_internal", instead pass it to Worker::new
- move "CliState::permissions" to "OpState"
- move "CliState::main_module" to "OpState"
- move "CliState::global_state" to "OpState"
- move "CliState::check_unstable()" to "GlobalState"
- change "cli_state()" to "global_state()"
- change "deno_core::ModuleLoader" trait to pass "OpState" to callbacks
- rename "CliState" to "CliModuleLoader"
This reverts the changes introduced by PR #5276, which made
`deno install «script»` automatically bundle the script's dependencies.
It broke the `deno install` command for a large number of scripts.
This reverts commit 34e98fa59c.
Closes: #7492
This commit adds basic support for collecting coverage
data using "deno test".
Currently the report is only a text added to the end
of output from "deno test".
Removes:
- "deno_core::StartupData"
- "deno_core::Script"
- "deno_core::OwnedScript"
Changes to "JsRuntime":
- remove "new_with_loader()"
- remove "with_heap_limits()"
- rename "IsolateOptions" to "RuntimeOptions" and make public
- "JsRuntime::new()" takes "RuntimeOptions" as a single param
This commit changes "deno info" subcommand logic.
- Modules are no longer loaded into V8 isolate - analysis
is done using ModuleGraph.
- Removed deno_core::Deps structure.
- Modules are no longer type-checked and transpiled -
"compiled" file is shown only if it is already available.
- Added number of unique dependencies for root module.
- Changed tree output:
- file size is shown next to the dependency
- repeated dependencies are marked with "*"
- used less spaces in prefix to save terminal width
deno_core/
- rename core_isolate.rs to runtime.rs
- rename CoreIsolate to JsRuntime
- rename JSError to JsError
- rename JSStackFrame to JsStackFrame
cli/
- update references from deno_core::CoreIsolate to deno_core::JsRuntime
- rename deno_core::JSError to deno_core::JsError
- rename fmt_errors::JSError to fmt_errors::JsError
- Add more support for generics
- Add the --private flag - displays documentation for
not exported and private nodes
- Display more attributes like abstract, static and readonly
- Display type aliases
- Refactor module to use the Display trait
- Use a bit more color
This PR removes direct dependency on dprint-plugin-typescript
and swc_ecma_visit. Both of these dependencies have been
pushed out to deno_lint crate. This should make it a bit easier
to do the upgrades and prevent having conflicting crate requirements.
* refactor "compile" and "runtimeCompile" in "compiler.ts" and factor out
separate methods for "compile" and "bundle" operations
* remove noisy debug output from "compiler.ts"
* provide "Serialize" implementations for enums in "msg.rs"
* rename "analyze_dependencies_and_references" to "pre_process_file" and
move it to "tsc.rs"
* refactor ModuleGraph to use more concrete types and properly annotate
locations where errors occur
* remove dead code from "file_fetcher.rs" - "SourceFile.types_url" is no
longer needed, as type reference parsing is done in "ModuleGraph"
* remove unneeded field "source_path" from ".meta" files stored for
compiled source file (towards #6080)
This commit:
* added default file globs so "deno lint" can be run
without arguments (just like "deno fmt")
* added test for globs in "deno lint"
* upgrade "deno_lint" crate to v0.1.9
This PR addresses many problems with module graph loading
introduced in #5029, as well as many long standing issues.
"ModuleGraphLoader" has been wired to "ModuleLoader" implemented
on "State" - that means that dependency analysis and fetching is done
before spinning up TS compiler worker.
Basic dependency tracking for TS compilation has been implemented.
Errors caused by import statements are now annotated with import
location.
Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
This commit completely overhauls how module analysis is
performed in TS compiler by moving the logic to Rust.
In the current setup module analysis is performed using
"ts.preProcessFile" API in a special TS compiler worker
running on a separate thread.
"ts.preProcessFile" allowed us to build a lot of functionality
in CLI including X-TypeScript-Types header support
and @deno-types directive support. Unfortunately at the
same time complexity of the ops required to perform
supporting tasks exploded and caused some hidden
permission escapes.
This PR introduces "ModuleGraphLoader" which can parse
source and load recursively all dependent source files; as
well as declaration files. All dependencies used in TS
compiler and now fetched and collected upfront in Rust
before spinning up TS compiler.
To achieve feature parity with existing APIs this commit
includes a lot of changes:
* add "ModuleGraphLoader"
- can fetch local and remote sources
- parses source code using SWC and extracts imports, exports, file references, special
headers
- this struct inherited all of the hidden complexity and cruft from TS version and requires
several follow up PRs
* rewrite cli/tsc.rs to perform module analysis upfront and send all required source code to
TS worker in one message
* remove op_resolve_modules and op_fetch_source_files from cli/ops/compiler.rs
* run TS worker on the same thread
This ensures the deno executable is properly created before running the integration tests.
Also allows deno_cli to be used as a lib. Docs are now properly generated: https://docs.rs/deno_cli/0.18.4/deno_cli/
Towards #2933
Prep for #2955
- Fixes cargo publish on deno_typescript, deno_cli_snapshots, and
deno_cli.
- Combines cli_snapshots and js into one directory.
- Extracts TS version at compile time rather than runtime
- Bumps version awkwardly - it was necessary to test end-to-end
publishing. Sorry.
- Adds git submodule deno_typescript/typescript
