This PR removes tokio_util::block_on - refactored compiler and file
fetcher slightly so that we can safely block there - that's because
only blocking path consist of only synchronous operations.
Additionally I removed excessive use of tokio_util::panic_on_error
and tokio_util::run_in_task and moved both functions to cli/worker.rs,
to tests module.
Closes #2960
After landing #3358 the benchmarks exploded indicating problems with workers and deno_core_http_bench.
This PR dramatically fixes thread/syscall count that showed up on benchmarks. Thread count is not back to previous levels but difference went from hundreds/thousands to about ~50.
- removes global `RESOURCE_TABLE` - resource tables are now created per `Worker`
in `State`
- renames `CliResource` to `StreamResource` and moves all logic related
to it to `cli/ops/io.rs`
- removes `cli/resources.rs`
- adds `state` argument to `op_read` and `op_write` and consequently adds
`stateful_minimal_op` to `State`
- IMPORTANT NOTE: workers don't have access to process stdio - this is
caused by fact that dropping worker would close stdout for process
(because it's constructed from raw handle, which closes underlying file
descriptor on drop)
This patch does not work with the recent bundler changes (#3325).
Unfortunately I didn't merge master before landing this patch. It has
something to do with console.log not working inside the compiler worker.
This reverts commit fd62379eaf.
- removes global `RESOURCE_TABLE` - resource tables are now created per `Worker`
in `State`
- renames `CliResource` to `StreamResource` and moves all logic related
to it to `cli/ops/io.rs`
- removes `cli/resources.rs`
- adds `state` argument to `op_read` and `op_write` and consequently adds
`stateful_minimal_op` to `State`
- IMPORTANT NOTE: workers don't have access to process stdio - this is
caused by fact that dropping worker would close stdout for process
(because it's constructed from raw handle, which closes underlying file
descriptor on drop)
- Bundles are fully standalone. They now include the shared loader with
`deno_typescript`.
- Refactor of the loader in `deno_typescript` to perform module
instantiation in a more
- Change of behaviour when an output file is not specified on the CLI.
Previously a default name was determined and the bundle written to that
file, now the bundle will be sent to `stdout`.
- Refactors in the TypeScript compiler to be able to support the concept
of a request type. This provides a cleaner abstraction and makes it
easier to support things like single module transpiles to the userland.
- Remove a "dangerous" circular dependency between `os.ts` and `deno.ts`,
and define `pid` and `noColor` in a better way.
- Don't bind early to `console` in `repl.ts`.
- Add an integration test for generating a bundle.
Towards simplifying (or better removing entirely) the CoreResource
trait. Resources should be any bit of privileged heap allocated memory
that needs to be referenced from JS, not very specific trait
implementations. Therefore CoreResource should be pushed towards being
as general as possible.
* Split ThreadSafeState into State and GlobalState. State is a "local"
state belonging to "Worker" while "GlobalState" is state shared by
whole program.
* Update "Worker" and ops to use "GlobalState" where applicable
* Move and refactor "WorkerChannels" resource
This avoids a crash when the Deno process has been running for
2**32 ms (about 50 days). Additionaly, time complexity of finding which
timer is due to fire next is reduced from from O(n) to O(log n).
Worth noting due to implementation of the Headers class the contents of headersMap have lowercase keys, although this matches the specification as header keys are case agnostic it does seem to not match behaviour of other implementations in other languages I have seen, would require some rewriting of Headers.ts
Basically this does pre-processing of TypeScript files and gathers all the
dependencies asynchronously. Only then after all the dependencies are gathered,
does it do a compile, which at that point all the dependencies are cached in
memory in the compiler, so with the exception of the hard coded assets, there
are no ops during the compilation.
Because op_fetch_source_files is now handled asynchronously in the runtime, we
can eliminate the tokio_util::block_on() which was causing the increase in
threads. Benchmarking on my machine has shown about a 5% improvement in speed
when dealing with compiling TypeScript. Still a long way to go, but an
improvement.
In theory the module name resolution and the fetching of the source files could
be broken out as two different ops. This would prevent situations of sending the
full source file all the time when actually the module is the same module
referenced by multiple modules, but that could be done subsequently to this.
