This commit rewrites file watcher used with --watch flag.
Instead of creating new watcher after each restart, only a single
watcher is created for whole process. Additionally debouncing
mechanism has been added to prevent infinite restart loops
if multiple files were changed in quick succession.
Co-authored-by: bartossh <lenart.consulting@gmail.com>
Previously, entering a single ']' would cause repl to forever accepting
new lines, due to that `ValidationResult::Invalid` would actually be
consumed by the editor itself while continue building the lines. Instead
we should mark it as `Valid` and send the bad input for evaluation to
get the proper error from V8.
Before:
```
> ]
(you can keep entering new line here, and it will never consume input
until you Ctrl-C)
```
After:
```
> ]
Uncaught SyntaxError: Unexpected token ']'
>
```
This writes all evaluaton results to stdout regardless if the result is
an error or not.
This matches the behavior of other read-eval-print-loops like Node.
This commit fixes test runner by awaitning "Deno.runTests()" call,
which ensures proper error is returned when there's an unresolved
promise that's being awaited.
This commit changes implementation of top-level-await in "deno_core".
Previously promise returned from module evaluation was not awaited,
leading to out-of-order execution of modules that have TLA. It's been
fixed by changing "JsRuntime::mod_evaluate" to be an async function
that resolves when the promise returned from module evaluation also
resolves. When waiting for promise resolution event loop is polled
repeatedly, until there are no more dynamic imports or pending
ops.
* Revert "refactor: Worker is not a Future (#7895)"
This reverts commit f4357f0ff9.
* Revert "refactor(core): JsRuntime is not a Future (#7855)"
This reverts commit d8879feb8c.
* Revert "fix(core): module execution with top level await (#7672)"
This reverts commit c7c7677825.
This commit fixes implementation of top level await in "deno_core".
Previously promise returned from module execution was ignored causing to execute
modules out-of-order.
With this commit promise returned from module execution is stored on "JsRuntime"
and event loop is polled until the promise resolves.
This makes use of a default referrer when its empty in repl mode so that
dynamic imports work in the global evaluation context.
Co-authored-by: Bartek Iwanczuk <biwanczuk@gmail.com>
This ports the REPL over to Rust and makes use of an inspector session to run a REPL on top of any isolate which lets make full use of rustylines various things like validators and completors without having to introduce a bunch of hard to test internal ops and glue code.
An accidental but good side effect of this is that the multiple line input we previously had is now an editable multi-line input prompt that is correctly stored in the history as a single entry.
This commit adds basic support for collecting coverage
data using "deno test".
Currently the report is only a text added to the end
of output from "deno test".
This commit changes "deno info" subcommand logic.
- Modules are no longer loaded into V8 isolate - analysis
is done using ModuleGraph.
- Removed deno_core::Deps structure.
- Modules are no longer type-checked and transpiled -
"compiled" file is shown only if it is already available.
- Added number of unique dependencies for root module.
- Changed tree output:
- file size is shown next to the dependency
- repeated dependencies are marked with "*"
- used less spaces in prefix to save terminal width
All benchmarks are done in Rust and can be invoked with
`cargo bench`.
Currently this has it's own "harness" that behaves like
`./tools/benchmark.py` did.
Because of this tests inside `cli/bench` are currently not run.
This should be switched to the language provided harness
once the `#[bench]` attribute has been stabilized.
This commit adds a fallback mechanism for absent compiled source file.
Because imported type declaration files are not emitted by TS compiler
and their imports are not elided users often hit "No such file or directory"
error. With this commit in such situation an empty source file will be
provided to V8 with a warning to the user suggesting using "import type"/
"export type" syntax instead.
This commit adds a "--no-check" option to following subcommands:
- "deno cache"
- "deno info"
- "deno run"
- "deno test"
The "--no-check" options allows to skip type checking step and instead
directly transpiles TS sources to JS sources.
This solution uses `ts.transpileModule()` API and is just an interim
solution before implementing it fully in Rust.
This commit adds incremental compilation capabilities to internal TS compiler.
Instead of using "ts.createProgram()" API for compilation step (during deno
startup), "ts.createIncrementalProgram()" API is used instead.
Thanks to TS' ".tsbuildinfo" file that already stores all necessary metadata
for compilation I was able to remove our own invention that is ".graph" file.
".tsbuildinfo" file is stored alongside compiled source and is used to
cache-bust outdated dependencies, facilitated by the "version" field.
The value for "version" field is computed in Rust during loading of module
graph and is basically a hash of the file contents.
Please keep in mind that incremental compilation is only used for initial
compilation (or dynamic imports compilation) - bundling and runtime compiler
APIs haven't been changed at all.
Due to problems with source map I changed compilation settings to inline
source map (inlineSourceMap instead of sourceMap).
Currently WebAssembly runtime errors don't propagate up to the user as
they use urls to denote where the error occurred which get caught by the source-map
pipeline which doesn't support the wasm scheme.
This commit:
* added default file globs so "deno lint" can be run
without arguments (just like "deno fmt")
* added test for globs in "deno lint"
* upgrade "deno_lint" crate to v0.1.9
This commit fixes several regressions in TS compiler:
* double compilation of same module during same process run
* compilation of JavaScript entry point with non-JS imports
* unexpected skip of emit during compilation
Additional checks were added to ensure "allowJs" setting is
used in TS compiler if JavaScript has non-JS dependencies.
This commit fixes regression that caused TS dependencies
not being compiled.
Check was added that ensures TS compiler is run if
any of dependencies in module graph is TS/TSX/JSX.
This PR addresses many problems with module graph loading
introduced in #5029, as well as many long standing issues.
"ModuleGraphLoader" has been wired to "ModuleLoader" implemented
on "State" - that means that dependency analysis and fetching is done
before spinning up TS compiler worker.
Basic dependency tracking for TS compilation has been implemented.
Errors caused by import statements are now annotated with import
location.
Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
Since everything that Deno loads is treated as an ES Module,
it means that all code is treated as "use strict" except for
when using the REPL. This PR changes that so code in the
REPL is also always evaluated with "use strict". There are
also a couple other places where we load code as scripts
which should also use "use strict" just in case.
This commit changes how error occurring in SWC are handled.
Changed lexer settings to properly handle TS decorators.
Changed output of SWC error to annotate with position in file.
This commit fixes a bug introduced in #5029 that caused bad
handling of redirects during module analysis.
Also ensured that duplicate modules are not downloaded.
This commit completely overhauls how module analysis is
performed in TS compiler by moving the logic to Rust.
In the current setup module analysis is performed using
"ts.preProcessFile" API in a special TS compiler worker
running on a separate thread.
"ts.preProcessFile" allowed us to build a lot of functionality
in CLI including X-TypeScript-Types header support
and @deno-types directive support. Unfortunately at the
same time complexity of the ops required to perform
supporting tasks exploded and caused some hidden
permission escapes.
This PR introduces "ModuleGraphLoader" which can parse
source and load recursively all dependent source files; as
well as declaration files. All dependencies used in TS
compiler and now fetched and collected upfront in Rust
before spinning up TS compiler.
To achieve feature parity with existing APIs this commit
includes a lot of changes:
* add "ModuleGraphLoader"
- can fetch local and remote sources
- parses source code using SWC and extracts imports, exports, file references, special
headers
- this struct inherited all of the hidden complexity and cruft from TS version and requires
several follow up PRs
* rewrite cli/tsc.rs to perform module analysis upfront and send all required source code to
TS worker in one message
* remove op_resolve_modules and op_fetch_source_files from cli/ops/compiler.rs
* run TS worker on the same thread
This commit removes "check_stderr" setting from itest! macro used
to generate integration tests. Without this setting on tests discarded
output of stderr making it very hard to debug the problem in test.
Numerous tests were changed by adding "--quiet" flag to not display
"Compile"/"Download" prompts.
This PR hot-fixes permission escapes in dynamic imports, workers
and runtime compiler APIs.
"permissions" parameter was added to public APIs of SourceFileFetcher
and appropriate permission checks are performed during loading of
local and remote files.
Importing .wasm files is non-standardized therefore deciding to
support current functionality past 1.0 release is risky.
Besides that .wasm import posed many challenges in our codebase
due to complex interactions with TS compiler which spawned
thread for each encountered .wasm import.
This commit removes:
- cli/compilers/wasm.rs
- cli/compilers/wasm_wrap.js
- two integration tests related to .wasm imports
The issue is solved by proxying websocket messages over a pair of
`futures::mpsc::unbounded` channels. As these are are implemented in
the 'futures' crate, they can't participate in Tokio's cooperative
task yielding.
This PR removes the hack in CLI that allows to run scripts with shorthand: deno script.ts.
Removing this functionality because it hacks around short-comings of clap our CLI parser. We agree that this shorthand syntax is desirable, but it needs to be rethinked and reimplemented. For 1.0 we should go with conservative approach that is correct.
Fixes #4000 and fixes #4476. Now always tries to fetch reference types
for JS files. Does not throw if it fails, since Typescript compiler will
complain if the file is not there(it will try to fetch it again first)
and people who just use JS should not be bothered by this error.
Not sure about my test, it passes and catches the bug but maybe there is
a better way to express it.
This is a first pass implementation which is still missing several important
features:
- support for --inspect-brk (#4503)
- support for source maps (#4501)
- support for piping console.log to devtools console (#4502)
Co-authored-by: Bert Belder <bertbelder@gmail.com>
Co-authored-by: Matt Harrison <mt.harrison86@gmail.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
* Remove DENO_BUILD_MODE and DENO_BUILD_PATH
Also remove outdated docs related to ninja/gn.
* fix
* remove parameter to build_mode()
* remove arg parsing from benchmark.py
After splitting "failFast" and "exitOnFail" arguments, there was a situation where failing tests did not exit with code 1.
* fixed argument value passed to Deno.runTests() in deno test
* fixed argument value passed to Deno.runTests() in std/testing/runner.ts
* added integration tests for deno test to ensure failFast and exitOnFail work as expected
* don't write test file to file system, but keep it in memory
Fixes #4101
Previously, we would just provide the raw JSON to the TypeScript
compiler worker, but TypeScript does not transform JSON. This caused
a problem when emitting a bundle, that the JSON would just be "inlined"
into the output, instead of being transformed into a module.
This fixes this problem by providing the compiled JSON to the TypeScript
compiler, so TypeScript just sees JSON as a "normal" TypeScript module.
* drop server guard before unit test result check
To prevent cascading test failures when js_unit_test http server
guard is dropped before asserting that tests were successful.
This is really a band-aid and doesn't solve underlying issue with
http server.
* Update CLI for unit_test_runner.ts
* Change cli/js/tests/unit_test_runner.ts command line interface to work in 3
modes:
- "one-off" - run tests that match permissions of currently running
process
- "master" - run tests for all possible permission combinations, by
spawning subprocesses running in "worker" mode and communicating via
TCP socket; requires elevated permissions
- "worker" - run tests for set of permissions provided by CLI arg;
requires elevated permissions to setup TCP connection to "master";
after initial setup process drops permissions to given set
* Support filtering of tests by string passed after "--" CLI arg
* Update cli/js/tests/README.md
Rewrites "cli/js/unit_test_runner.ts" to communicate with spawned subprocesses
using TCP socket.
* Rewrite "Deno.runTests()" by factoring out testing logic to private "TestApi"
class. "TestApi" implements "AsyncIterator" that yields "TestEvent"s,
which is an interface for different types of event occuring during running
tests.
* Add "reporter" argument to "Deno.runTests()" to allow users to provide custom
reporting mechanism for tests. It's represented by "TestReporter" interface,
that implements hook functions for each type of "TestEvent". If "reporter"
is not provided then default console reporting is used (via
"ConsoleReporter").
* Change how "unit_test_runner" communicates with spawned suprocesses. Instead
of parsing text data from child's stdout, a TCP socket is created and used
for communication. "unit_test_runner" can run in either "master" or "worker"
mode. Former is responsible for test discovery and establishing needed
permission combinations; while latter (that is spawned by "master") executes
tests that match given permission set.
* Use "SocketReporter" that implements "TestReporter" interface to send output
of tests to "master" process. Data is sent as stringified JSON and then
parsed by "master" as structured data. "master" applies it's own reporting
logic to output tests to console (by reusing default "ConsoleReporter").
