3e98ea4e69
Allows writing named sub-tests. These are: 1. Filterable on the command line via `cargo test ...` 2. Run in parallel 3. Use a fresh temp and deno dir for each test (unlike steps)
3.1 KiB
specs
These are integration tests that execute the deno binary. They supersede the
itest macro found in the tests/integration folder and are the preferred way
of writing tests that use the deno binary.
Structure
Tests must have the following directory structure:
tests/specs/<category_name>/<test_name>/__test__.json
Test filtering
To run a specific test, run:
cargo test specs::category_name::test_name
Or just the following, though it might run other tests:
cargo test test_name
__test__.json file
This file describes the test(s) to execute and the steps to execute. A basic example looks like:
{
"args": "run main.js",
"output": "main.out"
}
This will run deno run main.js then assert that the output matches the text in
main.out.
Or another example that runs multiple steps:
{
"tempDir": true,
"steps": [{
"args": "cache main.ts",
"output": "cache.out"
}, {
"args": "run main.ts",
"output": "error.out",
"exitCode": 1
}]
}
Or if you want to run several tests at the same time:
{
"tests": {
"ignore_dir": {
"args": "run script.ts",
"output": "script.out"
},
"some_other_test": {
"args": "run other.ts",
"output": "other.out"
}
}
}
Top level properties
base- The base config to use for the test. Options:jsr- Uses env vars for jsr.npm- Uses env vars for npm.
tempDir(boolean) - Copy all the non-test files to a temporary directory and execute the command in that temporary directory.- By default, tests are executed with a current working directory of the test, but this may not be desirable for tests such as ones that create a node_modules directory.
Step properties
When writing a single step, these may be at the top level rather than nested in a "steps" array or "tests" object.
args- A string (that will be spilt on whitespace into an args array) or an array of arguments.output- Path to use to assert the output or text (must end with an .out extension) or text to pattern match against the output.flaky- Step should be repeated until success a maximum of 3 times.if("windows","linux","mac","unix") - Whether to run this step.exitCode(number) - Expected exit code.
Auto-complete
To get auto-complete for these files, add the following to a local
.vscode/settings.json file:
{
"json.schemas": [{
"fileMatch": [
"__test__.jsonc"
],
"url": "./tests/specs/schema.json"
}]
}
.out files
.out files are used to assert the output when running a test or test step.
Within the file, you can use the following for matching:
[WILDCARD]- match any text at the wildcard[WILDLINE]- match any text on the current line[WILDCHAR]- match the next character[WILDCHARS(5)]- match any of the next 5 characters[UNORDERED_START]followed by many lines then[UNORDERED_END]will match the lines in any order (useful for non-deterministic output)[# example]- line comments start with[#and end with]