mirror of
https://github.com/denoland/deno.git
synced 2024-12-21 23:04:45 -05:00
a2a537e196
Fixes #23053. Two small bugs here: - the existing condition for printing out the group header was broken. it worked in the reproducer (in the issue above) without filtering only by accident, due to setting `self.has_ungrouped = true` once we see the warmup bench. Knowing that we sort benchmarks to put ungrouped benches first, there are only two cases: 1) we are starting the first group 2) we are ending the previous group and starting a new group - when you passed `--filter` we were applying that filter to the warmup bench (which is not visible to users), so we suffered from jit bias if you were filtering (unless your filter was `<warmup>`) TLDR; Running ```bash deno bench main.js --filter="G" ``` ```js // main.js Deno.bench({ group: "G1", name: "G1-A", fn() {}, }); Deno.bench({ group: "G1", name: "G1-B", fn() {}, }); ``` Before this PR: ``` benchmark time (avg) iter/s (min … max) p75 p99 p995 --------------------------------------------------------------- ----------------------------- G1-A 303.52 ps/iter3,294,726,102.1 (254.2 ps … 7.8 ns) 287.5 ps 391.7 ps 437.5 ps G1-B 3.8 ns/iter 263,360,635.9 (2.24 ns … 8.36 ns) 3.84 ns 4.73 ns 4.94 ns summary G1-A 12.51x faster than G1-B ``` After this PR: ``` benchmark time (avg) iter/s (min … max) p75 p99 p995 --------------------------------------------------------------- ----------------------------- group G1 G1-A 3.85 ns/iter 259,822,096.0 (2.42 ns … 9.03 ns) 3.83 ns 4.62 ns 4.83 ns G1-B 3.84 ns/iter 260,458,274.5 (3.55 ns … 7.05 ns) 3.83 ns 4.45 ns 4.7 ns summary G1-B 1x faster than G1-A ``` |
||
|---|---|---|
| .. | ||
| bench | ||
| check/types_resolved_relative_config | ||
| coverage | ||
| flags | ||
| future/runtime_api | ||
| import_map/import_map_config | ||
| info/ts_error | ||
| jsr | ||
| jupyter/install_command_not_exists | ||
| lint | ||
| node/detect_es_module_defined_as_cjs | ||
| npm | ||
| run/redirect_javascript | ||
| test | ||
| mod.rs | ||
| README.md | ||
specs
These are integration tests that execute the deno binary. They supersede the
itest macro found in the tests/integration folder and are the preferred way
of writing tests that use the deno binary.
Structure
Tests must have the following directory structure:
tests/specs/<category_name>/<test_name>/__test__.json
Test filtering
To run a specific test, run:
cargo test specs::category_name::test_name
Or just the following, though it might run other tests:
cargo test test_name
__test__.json file
This file describes the test to execute and the steps to execute. A basic example looks like:
{
"args": "run main.js",
"output": "main.out"
}
This will run deno run main.js then assert that the output matches the text in
main.out.
Or another example that runs multiple steps:
{
"tempDir": true,
"steps": [{
"args": "cache main.ts",
"output": "cache.out"
}, {
"args": "run main.ts",
"output": "error.out",
"exitCode": 1
}]
}
Top level properties
base- The base config to use for the test. Options:jsr- Uses env vars for jsr.npm- Uses env vars for npm.
tempDir(boolean) - Copy all the non-test files to a temporary directory and execute the command in that temporary directory.- By default, tests are executed with a current working directory of the test, but this may not be desirable for tests such as ones that create a node_modules directory.
Step properties
When writing a single step, these may be at the top level rather than nested in a "steps" array.
args- A string (that will be spilt on whitespace into an args array) or an array of arguments.output- Path to use to assert the output.clean(boolean) - Whether to empty the deno_dir before running the step.exitCode(number) - Expected exit code.
.out files
.out files are used to assert the output when running a test or test step.
Within the file, you can use the following for matching:
[WILDCARD]- match any text at the wildcard[WILDLINE]- match any text on the current line[WILDCHARS(5)]- match any of the next 5 characters[UNORDERED_START]followed by many lines then[UNORDERED_END]will match the lines in any order (useful for non-deterministic output)[# example]- line comments start with[#and end with]