This PR removes op_cache and refactors how Deno interacts with TS compiler.
Ultimate goal is to completely sandbox TS compiler worker; it should operate on
simple request -> response basis. With this commit TS compiler no longer
caches compiled sources as they are generated but rather collects all sources
and sends them back to Rust when compilation is done.
Additionally "Diagnostic" and its children got refactored to use "Deserialize" trait
instead of manually implementing JSON deserialization.
- Cleanup "tsCompilerOnMessage" by factoring out separate methods for each
request type:
* "compile"
* "runtimeCompile"
* "runtimeTranspile"
- Simplify control flow of compiler workers by a) no longer calling "close()" in worker runtime after a
single message; b) explicitly shutting down worker from host after a single message
Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
To better reflect changes in error types in JS from #3662 this PR changes
default error type used in ops from "ErrBox" to "OpError".
"OpError" is a type that can be sent over to JSON; it has all
information needed to construct error in JavaScript. That
made "GetErrorKind" trait useless and so it was removed altogether.
To provide compatibility with previous use of "ErrBox" an implementation of
"From<ErrBox> for OpError" was added, however, it is an escape hatch and
ops implementors should strive to use "OpError" directly.
* Use PathBuf for DenoSubcommand::Bundle's out_file
* Use PathBuf for DenoSubcommand::Format's files
* Use PathBuf for DenoSubcommand::Install's dir
* Use PathBuf for read/write whitelists
* establish basic event loop for workers
* make "self.close()" inside worker
* remove "runWorkerMessageLoop() - instead manually call global function
in Rust when message arrives. This is done in preparation for structured clone
* refactor "WorkerChannel" and use distinct structs for internal
and external channels; "WorkerChannelsInternal" and "WorkerHandle"
* move "State.worker_channels_internal" to "Worker.internal_channels"
* add "WorkerEvent" enum for child->host communication;
currently "Message(Buf)" and "Error(ErrBox)" variants are supported
* add tests for nested workers
* add tests for worker throwing error on startup
This change simplifies how we execute V8. Previously V8 Isolates jumped
around threads every time they were woken up. This was overly complex and
potentially hurting performance in a myriad ways. Now isolates run on
their own dedicated thread and never move.
- blocking_json spawns a thread and does not use a thread pool
- op_host_poll_worker and op_host_resume_worker are non-operational
- removes Worker::get_message and Worker::post_message
- ThreadSafeState::workers table contains WorkerChannel entries instead
of actual Worker instances.
- MainWorker and CompilerWorker are no longer Futures.
- The multi-threaded version of deno_core_http_bench was removed.
- AyncOps no longer need to be Send + Sync
This PR is very large and several tests were disabled to speed
integration:
- installer_test_local_module_run
- installer_test_remote_module_run
- _015_duplicate_parallel_import
- _026_workers
* split ops/worker.rs into ops/worker_host.rs and ops/web_worker.rs
* refactor js/workers.ts and factor out js/worker_main.ts - entry point for WebWorker runtime
* BREAKING CHANGE: remove support for blob: URL in Worker
* BREAKING CHANGE: remove Deno namespace support and noDenoNamespace option in Worker constructor
* introduce WebWorker struct which is a stripped down version of cli::Worker
tokio_util::run and tokio::run_on_current_thread should accept Future<Output=()> instead of Future<Output=Result<(), ()>>. Currently, all the passed futures have to add Ok(()) or futures::future::ok(()) unnecessarily to call this method.
This PR removes tokio_util::block_on - refactored compiler and file
fetcher slightly so that we can safely block there - that's because
only blocking path consist of only synchronous operations.
Additionally I removed excessive use of tokio_util::panic_on_error
and tokio_util::run_in_task and moved both functions to cli/worker.rs,
to tests module.
Closes #2960
After landing #3358 the benchmarks exploded indicating problems with workers and deno_core_http_bench.
This PR dramatically fixes thread/syscall count that showed up on benchmarks. Thread count is not back to previous levels but difference went from hundreds/thousands to about ~50.
