This commit completely overhauls how module analysis is
performed in TS compiler by moving the logic to Rust.
In the current setup module analysis is performed using
"ts.preProcessFile" API in a special TS compiler worker
running on a separate thread.
"ts.preProcessFile" allowed us to build a lot of functionality
in CLI including X-TypeScript-Types header support
and @deno-types directive support. Unfortunately at the
same time complexity of the ops required to perform
supporting tasks exploded and caused some hidden
permission escapes.
This PR introduces "ModuleGraphLoader" which can parse
source and load recursively all dependent source files; as
well as declaration files. All dependencies used in TS
compiler and now fetched and collected upfront in Rust
before spinning up TS compiler.
To achieve feature parity with existing APIs this commit
includes a lot of changes:
* add "ModuleGraphLoader"
- can fetch local and remote sources
- parses source code using SWC and extracts imports, exports, file references, special
headers
- this struct inherited all of the hidden complexity and cruft from TS version and requires
several follow up PRs
* rewrite cli/tsc.rs to perform module analysis upfront and send all required source code to
TS worker in one message
* remove op_resolve_modules and op_fetch_source_files from cli/ops/compiler.rs
* run TS worker on the same thread
Importing .wasm files is non-standardized therefore deciding to
support current functionality past 1.0 release is risky.
Besides that .wasm import posed many challenges in our codebase
due to complex interactions with TS compiler which spawned
thread for each encountered .wasm import.
This commit removes:
- cli/compilers/wasm.rs
- cli/compilers/wasm_wrap.js
- two integration tests related to .wasm imports
This PR removes op_cache and refactors how Deno interacts with TS compiler.
Ultimate goal is to completely sandbox TS compiler worker; it should operate on
simple request -> response basis. With this commit TS compiler no longer
caches compiled sources as they are generated but rather collects all sources
and sends them back to Rust when compilation is done.
Additionally "Diagnostic" and its children got refactored to use "Deserialize" trait
instead of manually implementing JSON deserialization.
refactor: Parse URLs more sequentially. This makes it easier to change matching behaviour depending on the protocol.
fix: Fail when a host isn't given for certain protocols.
fix: Convert back-slashes info forward-slashes.
Keep in mind Buffer.toString() still exists, but returns [object Object].
Reason for removal of Buffer.toString() was that it implicitly used
TextDecoder with fixed "utf-8" encoding and no way to customize
the encoding.
This change is to prevent needed a separate stat syscall for each file
when using readdir.
For consistency, this PR also modifies std's `WalkEntry` interface to
extend `DirEntry` with an additional `path` field.
This commit removes "combined" interfaces from cli/js/io.ts; in the
like of "ReadCloser", "WriteCloser" in favor of using intersections
of concrete interfaces.
Changed `URL.port` implementation to match [WHATWG
specifications](https://url.spec.whatwg.org/#port-state).
This PR matches the behaviour of other browsers:
1. a `TypeError` must be thrown when passing an URL with an invalid
port to the constructor.
2. When setting an invalid port, using property setter, I haven't found
what should happen in this case, so I mimic **Firefox** & **Node**
behaviour. If an invalid port is set, it will use the previous value.
**Chrome** sets the value to `'0'` if an invalid port is set. I prefer
to keep the previous valid value. (I can use Chrome's behaviour if you
think it's better, it's a simple value change)
```
url.port = '3000'; // valid
url.port = 'deno'; // invalid
assertEquals(url.port, '3000');
```
3. If the port value equals the current protocol default port value,
`port` will be an empty string.
When creating a console instance, one must pass "printFunc" arg
which is used internally by Console to output messages.
Due to numerous refactors there was a single method ("console.clear()")
that used "Deno.stdout" instead of "printFunc".
This commit unifies how "Console" outpus message, by using
"printFunc" in all methods; consequently "Deno.stdout" is no longer
imported in "cli/js/console.ts" making it a standalone module that doesn't
depend on any CLI-specific APIs.
Deno.runTests() interface is not yet good enough to be exposed
publicly with stability guarantees.
This commit removes public API related to testing: Deno.runTests()
and Deno.TestMessage, but keeps them exposed on Deno.internal object
so they can be used with "deno test" subcommand.
Refactors Event and EventTarget so that they better encapsulate their
non-public data as well as are more forward compatible with things like
DOM Nodes.
Moves `dom_types.ts` -> `dom_types.d.ts` which was always the intention,
it was a legacy of when we used to build the types from the code and the
limitations of the compiler. There was a lot of cruft in `dom_types`
which shouldn't have been there, and mis-alignment to the DOM standards.
This generally has been eliminated, though we still have some minor
differences from the DOM (like the removal of some deprecated
methods/properties).
Adds `DOMException`. Strictly it shouldn't inherit from `Error`, but
most browsers provide a stack trace when one is thrown, so the behaviour
in Deno actually better matches the browser.
`Event` still doesn't log to console like it does in the browser. I
wanted to get this raised and that could be an enhancement later on (it
currently doesn't either).
- Removes the __fetch namespace from `deno types`
- Response.redirect should be a static.
- Response.body should not be AsyncIterable.
- Disables the deno_proxy benchmark
- Makes std/examples/curl.ts buffer the body before printing to stdout
* Reduce "testing" interfaces
* Use a callback instead of a generator for Deno.runTests()
* Default RunTestsOptions::reportToConsole to true
* Compose TestMessage into a single interface
* Properly track isFile, isSymlink, isDirectory
These don't exhaust all the possibilities, so none of them should be
defined as "neither of the others".
* empty
1. Array elements are now grouped the same as in Node.js
2. Limit to 100 (Node.js default) elements to display in iterable
3. Print each element in new line if excessing max line length (same as in Node.js)
4. Print length of the TypedArray
5. Print information about empty items in Array
This a complex boring PR that shifts around code (primarily) in cli/fs.rs and
cli/ops/fs.rs. The gain of this refactoring is to ease the way for #4188 and
#4017, and also to avoid some future development pain.
Mostly there is no change in functionality. Except:
* squashed bugs where op_utime and op_chown weren't using `resolve_from_cwd`
* eliminated the use of the external `remove_dir_all` crate.
* op_chmod now only queries metadata to verify file/dir exists on Windows (it
will already fail on Unix if it doesn't)
* op_chown now verifies the file/dir's existence on Windows like chmod does.
* add tests for "Deno.core.encode" and "Deno.core.decode" for empty inputs
* use "Deno.core.encode" in "TextEncoder"
* use "Deno.core.decode" in "TextDecoder"
* remove "core_decode" and "core_encode" benchmarks
This PR brings assertOps and assertResources sanitizers to Deno.test() API.
assertOps checks that test doesn't leak async ops, ie. there are no unresolved
promises originating from Deno APIs. Enabled by default, can be disabled using
Deno.TestDefinition.disableOpSanitizer.
assertResources checks that test doesn't leak resources, ie. all resources used
in test are closed. For example; if a file is opened during a test case it must be
explicitly closed before test case finishes. It's most useful for asynchronous
generators. Enabled by default, can be disabled using
Deno.TestDefinition.disableResourceSanitizer.
We've used those sanitizers in internal runtime tests and it proved very useful in
surfacing incorrect tests which resulted in interference between the tests.
All tests have been sanitized.
Closes #4208
Fixes #4101
Previously, we would just provide the raw JSON to the TypeScript
compiler worker, but TypeScript does not transform JSON. This caused
a problem when emitting a bundle, that the JSON would just be "inlined"
into the output, instead of being transformed into a module.
This fixes this problem by providing the compiled JSON to the TypeScript
compiler, so TypeScript just sees JSON as a "normal" TypeScript module.
This PR attempts to fix intermittent errors occurring on Windows for "cli/tests/unit_test_runner.ts." Runner has been reworked to create only single TCP listener instead of one listener per worker.
Additionally worker doesn't close TCP socket - it waits for parent process to close the socket and only then exits.
This commit changes output of default test reporter to resemble output from Rust test runner;
first the name of running test is printed with "...", then after test has run result is printed on the same line.
* Split "Deno.TestEvent.Result" into "TestStart" and "TestEnd";
* changes TestReporter interface to support both events;
Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
This commits add two new methods to "Deno.core" namespace: "encode" and "decode".
Those methods are bound in Rust to provide a) fast b) generally available of encoding and decoding UTF-8 strings.
Both methods are now used in "cli/js/dispatch_json.ts".
* drop server guard before unit test result check
To prevent cascading test failures when js_unit_test http server
guard is dropped before asserting that tests were successful.
This is really a band-aid and doesn't solve underlying issue with
http server.
* Update CLI for unit_test_runner.ts
* Change cli/js/tests/unit_test_runner.ts command line interface to work in 3
modes:
- "one-off" - run tests that match permissions of currently running
process
- "master" - run tests for all possible permission combinations, by
spawning subprocesses running in "worker" mode and communicating via
TCP socket; requires elevated permissions
- "worker" - run tests for set of permissions provided by CLI arg;
requires elevated permissions to setup TCP connection to "master";
after initial setup process drops permissions to given set
* Support filtering of tests by string passed after "--" CLI arg
* Update cli/js/tests/README.md
