Moves op_close and op_resources to deno_core::ops and exports them.
Adds serde dependency to deno_core and reexports it.
Moves JS implementation of those ops to Deno.core and reexports them in Deno.
Removes:
- "deno_core::StartupData"
- "deno_core::Script"
- "deno_core::OwnedScript"
Changes to "JsRuntime":
- remove "new_with_loader()"
- remove "with_heap_limits()"
- rename "IsolateOptions" to "RuntimeOptions" and make public
- "JsRuntime::new()" takes "RuntimeOptions" as a single param
Provides a concrete state type that can be dynamically added. This is necessary for op crates.
* renames BasicState to OpState
* async ops take `Rc<RefCell<OpState>>`
* sync ops take `&mut OpState`
* removes `OpRegistry`, `OpRouter` traits
* `get_error_class_fn` moved to OpState
* ResourceTable moved to OpState
deno_core/
- rename core_isolate.rs to runtime.rs
- rename CoreIsolate to JsRuntime
- rename JSError to JsError
- rename JSStackFrame to JsStackFrame
cli/
- update references from deno_core::CoreIsolate to deno_core::JsRuntime
- rename deno_core::JSError to deno_core::JsError
- rename fmt_errors::JSError to fmt_errors::JsError
This PR addresses many problems with module graph loading
introduced in #5029, as well as many long standing issues.
"ModuleGraphLoader" has been wired to "ModuleLoader" implemented
on "State" - that means that dependency analysis and fetching is done
before spinning up TS compiler worker.
Basic dependency tracking for TS compilation has been implemented.
Errors caused by import statements are now annotated with import
location.
Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
This is a first pass implementation which is still missing several important
features:
- support for --inspect-brk (#4503)
- support for source maps (#4501)
- support for piping console.log to devtools console (#4502)
Co-authored-by: Bert Belder <bertbelder@gmail.com>
Co-authored-by: Matt Harrison <mt.harrison86@gmail.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
* establish basic event loop for workers
* make "self.close()" inside worker
* remove "runWorkerMessageLoop() - instead manually call global function
in Rust when message arrives. This is done in preparation for structured clone
* refactor "WorkerChannel" and use distinct structs for internal
and external channels; "WorkerChannelsInternal" and "WorkerHandle"
* move "State.worker_channels_internal" to "Worker.internal_channels"
* add "WorkerEvent" enum for child->host communication;
currently "Message(Buf)" and "Error(ErrBox)" variants are supported
* add tests for nested workers
* add tests for worker throwing error on startup
This change simplifies how we execute V8. Previously V8 Isolates jumped
around threads every time they were woken up. This was overly complex and
potentially hurting performance in a myriad ways. Now isolates run on
their own dedicated thread and never move.
- blocking_json spawns a thread and does not use a thread pool
- op_host_poll_worker and op_host_resume_worker are non-operational
- removes Worker::get_message and Worker::post_message
- ThreadSafeState::workers table contains WorkerChannel entries instead
of actual Worker instances.
- MainWorker and CompilerWorker are no longer Futures.
- The multi-threaded version of deno_core_http_bench was removed.
- AyncOps no longer need to be Send + Sync
This PR is very large and several tests were disabled to speed
integration:
- installer_test_local_module_run
- installer_test_remote_module_run
- _015_duplicate_parallel_import
- _026_workers
* split ops/worker.rs into ops/worker_host.rs and ops/web_worker.rs
* refactor js/workers.ts and factor out js/worker_main.ts - entry point for WebWorker runtime
* BREAKING CHANGE: remove support for blob: URL in Worker
* BREAKING CHANGE: remove Deno namespace support and noDenoNamespace option in Worker constructor
* introduce WebWorker struct which is a stripped down version of cli::Worker
* refactored RecursiveLoad - it was renamed to RecursiveModuleLoad, it does not take ownership of isolate anymore - a struct implementing Stream that yields SourceCodeInfo
* untangled module loading logic between RecursiveLoad and isolate - that logic is encapsulated in EsIsolate and RecursiveModuleLoad, where isolate just consumes modules as they become available - does not require to pass Arc<Mutex<Isolate>> around anymore
* removed EsIsolate.mods_ in favor of Modules and moved them inside EsIsolate
* EsIsolate now requires "loader" argument during construction - struct that implements Loader trait
* rewrite first methods on isolate as async
tokio_util::run and tokio::run_on_current_thread should accept Future<Output=()> instead of Future<Output=Result<(), ()>>. Currently, all the passed futures have to add Ok(()) or futures::future::ok(()) unnecessarily to call this method.
This PR removes tokio_util::block_on - refactored compiler and file
fetcher slightly so that we can safely block there - that's because
only blocking path consist of only synchronous operations.
Additionally I removed excessive use of tokio_util::panic_on_error
and tokio_util::run_in_task and moved both functions to cli/worker.rs,
to tests module.
Closes #2960
After landing #3358 the benchmarks exploded indicating problems with workers and deno_core_http_bench.
This PR dramatically fixes thread/syscall count that showed up on benchmarks. Thread count is not back to previous levels but difference went from hundreds/thousands to about ~50.
Towards simplifying (or better removing entirely) the CoreResource
trait. Resources should be any bit of privileged heap allocated memory
that needs to be referenced from JS, not very specific trait
implementations. Therefore CoreResource should be pushed towards being
as general as possible.
* Split ThreadSafeState into State and GlobalState. State is a "local"
state belonging to "Worker" while "GlobalState" is state shared by
whole program.
* Update "Worker" and ops to use "GlobalState" where applicable
* Move and refactor "WorkerChannels" resource
* merge SourceFileFetcher trait and FileFetcher struct
* move logic related to source file fetching to //cli/file_fetcher.rs
* use Result when creating new ThreadSafeState
* rename `ModuleMetaData` to `SourceFile` and remove TS specific
functionality
* add `TsCompiler` struct encapsulating processing of TypeScript files
* move `SourceMapGetter` trait implementation to `//cli/compiler.rs`
* add low-level `DiskCache` API for general purpose caches and use it in
`DenoDir` and `TsCompiler` for filesystem access
* don't use hash-like filenames for compiled modules, instead use
metadata file for storing compilation hash
* add `SourceFileCache` for in-process caching of loaded files for fast
subsequent access
* define `SourceFileFetcher` trait encapsulating loading of local and
remote files and implement it for `DenoDir`
* define `use_cache` and `no_fetch` flags on `DenoDir` instead of using
in fetch methods
The rules are now as follows:
* In `import` statements, as mandated by the WHATWG specification,
the import specifier is always treated as a URL.
If it is a relative URL, it must start with either / or ./ or ../
* A script name passed to deno as a command line argument may be either
an absolute URL or a local path.
- If the name starts with a valid URI scheme followed by a colon, e.g.
'http:', 'https:', 'file:', 'foo+bar:', it always interpreted as a
URL (even if Deno doesn't support the indicated protocol).
- Otherwise, the script name is interpreted as a local path. The local
path may be relative, and operating system semantics determine how
it is resolved. Prefixing a relative path with ./ is not required.
