This PR brings assertOps and assertResources sanitizers to Deno.test() API.
assertOps checks that test doesn't leak async ops, ie. there are no unresolved
promises originating from Deno APIs. Enabled by default, can be disabled using
Deno.TestDefinition.disableOpSanitizer.
assertResources checks that test doesn't leak resources, ie. all resources used
in test are closed. For example; if a file is opened during a test case it must be
explicitly closed before test case finishes. It's most useful for asynchronous
generators. Enabled by default, can be disabled using
Deno.TestDefinition.disableResourceSanitizer.
We've used those sanitizers in internal runtime tests and it proved very useful in
surfacing incorrect tests which resulted in interference between the tests.
All tests have been sanitized.
Closes #4208
Fixes #4101
Previously, we would just provide the raw JSON to the TypeScript
compiler worker, but TypeScript does not transform JSON. This caused
a problem when emitting a bundle, that the JSON would just be "inlined"
into the output, instead of being transformed into a module.
This fixes this problem by providing the compiled JSON to the TypeScript
compiler, so TypeScript just sees JSON as a "normal" TypeScript module.
This PR attempts to fix intermittent errors occurring on Windows for "cli/tests/unit_test_runner.ts." Runner has been reworked to create only single TCP listener instead of one listener per worker.
Additionally worker doesn't close TCP socket - it waits for parent process to close the socket and only then exits.
This commit changes output of default test reporter to resemble output from Rust test runner;
first the name of running test is printed with "...", then after test has run result is printed on the same line.
* Split "Deno.TestEvent.Result" into "TestStart" and "TestEnd";
* changes TestReporter interface to support both events;
Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
This commits add two new methods to "Deno.core" namespace: "encode" and "decode".
Those methods are bound in Rust to provide a) fast b) generally available of encoding and decoding UTF-8 strings.
Both methods are now used in "cli/js/dispatch_json.ts".
* drop server guard before unit test result check
To prevent cascading test failures when js_unit_test http server
guard is dropped before asserting that tests were successful.
This is really a band-aid and doesn't solve underlying issue with
http server.
* Update CLI for unit_test_runner.ts
* Change cli/js/tests/unit_test_runner.ts command line interface to work in 3
modes:
- "one-off" - run tests that match permissions of currently running
process
- "master" - run tests for all possible permission combinations, by
spawning subprocesses running in "worker" mode and communicating via
TCP socket; requires elevated permissions
- "worker" - run tests for set of permissions provided by CLI arg;
requires elevated permissions to setup TCP connection to "master";
after initial setup process drops permissions to given set
* Support filtering of tests by string passed after "--" CLI arg
* Update cli/js/tests/README.md
Rewrites "cli/js/unit_test_runner.ts" to communicate with spawned subprocesses
using TCP socket.
* Rewrite "Deno.runTests()" by factoring out testing logic to private "TestApi"
class. "TestApi" implements "AsyncIterator" that yields "TestEvent"s,
which is an interface for different types of event occuring during running
tests.
* Add "reporter" argument to "Deno.runTests()" to allow users to provide custom
reporting mechanism for tests. It's represented by "TestReporter" interface,
that implements hook functions for each type of "TestEvent". If "reporter"
is not provided then default console reporting is used (via
"ConsoleReporter").
* Change how "unit_test_runner" communicates with spawned suprocesses. Instead
of parsing text data from child's stdout, a TCP socket is created and used
for communication. "unit_test_runner" can run in either "master" or "worker"
mode. Former is responsible for test discovery and establishing needed
permission combinations; while latter (that is spawned by "master") executes
tests that match given permission set.
* Use "SocketReporter" that implements "TestReporter" interface to send output
of tests to "master" process. Data is sent as stringified JSON and then
parsed by "master" as structured data. "master" applies it's own reporting
logic to output tests to console (by reusing default "ConsoleReporter").
- moves signal definition from "cli/js/process.ts" to "cli/js/signals.ts"
- removes "cli/js/dispatch.ts"
- removes "cli/js/types.ts"
- moves web specific utilities to "cli/js/web/util.ts"
- moves compiler implementation to "cli/js/compiler/" directory
- moves more APIs to "cli/js/web":
* "console.ts"
* "console_table.ts"
* "performance.ts"
* "timers.ts"
* "workers.ts"
- removes some dead code from "cli/js/"
Listener and UDPConn are AsyncIterables instead of AsyncIterators.
The [Symbol.asyncIterator]()s are defined as generators and the
next() methods are gone.
"Listener/Socket has been closed" errors are now BadResource.
All Deno runtime test files were moved to cli/js/tests/ directory.
It makes a clear distinction that cli/js/tests/ contains code
that is run under Deno runtime as opposed to code in cli/js/ which
is used to create bundle and snapshot with "deno_typescript".
Following JS ops were moved to separate files in cli/js/ops directory:
- io
- process
- worker_host
- web_worker
- plugins
- timers
- signal
- permissions
Following JS ops were moved to separate files in cli/js/ops directory:
- compiler
- dispatch_json
- dispatch_minimal
- errors
- fetch
- fs_events
- os
- random
- repl
- resources
- runtime_compiler
- runtime
- tty
There's a lot of variation in doc comments and internal code about
whether chmod/0o777-style permissions are called `mode` or `perm`. (For
example, mkdir and writeFile choose differently.)
Had proposed earlier to go consistently with `perm`, but on balance devs
prefer to go with `mode`.
There's a lot of variation in doc comments and internal code about
whether the first parameter to file system calls is `path` or `name` or
`filename`. For consistency, have made it always be `path`.
* refactor: preliminary cleanup of Deno.runTests()
* Change time measurement to use new Date() instead of
performance.now(). Because there is no guarantee that tests are
run with "--allow-hr" using new Date() guarantees higher
precision of 1ms instead of 2ms.
* Support String type filter in "skip" and "only".
* Split "exitOnFail" into "exitOnFail" and "failFast".
Former tells if "runTests()" should exit with code 1 on test
failure, while latter tells if "runTests()" should stop
running tests on first failure.
* Use "defer" to wait for unhandled promise rejection - this bit
is funky and doesn't seem right, but for now it's just a rewrite
from using "setTimeout". Intended to be fixed in later commits.
* Remove global "__DENO_TEST_REGISTRY", don't expose list of
registered tests (to be addressed in follow up commits)
* Remove arbitrary slow test threshold; use uniform coloring
instead
- Cleanup "tsCompilerOnMessage" by factoring out separate methods for each
request type:
* "compile"
* "runtimeCompile"
* "runtimeTranspile"
- Simplify control flow of compiler workers by a) no longer calling "close()" in worker runtime after a
single message; b) explicitly shutting down worker from host after a single message
Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
Rewrite "testPerm" helper function used for testing of internal
runtime code. It's been renamed to "unitTest" and provides API that
is extensible in the future by accepting optional "UnitTestOptions"
argument. "test" helper was also removed and replaced by
overloaded version of "unitTest" that takes only function argument.
"UnitTestOptions" currently supports "perms" and "skip"
options, where former works exactly as first argument to "testPerm"
did, while the latter allows to conditionally skip tests.
* add "assertOps" test assertion which makes sure test case
is not "leaking" ops - ie. after test finishes there are no
pending async ops
* apply "assertOps" to all tests in "cli/js/"
* fix numerous tests leaking ops
* document problem with edge case in "clearInterval"
and "clearTimeout" implementation where they
may leak async ops
* move "cli/js/worker_test.ts" to "cli/tests/worker_test.ts" and
run as integration test; workers leak ops because of missing
"terminate" implementation
