mirror of
https://github.com/denoland/deno.git
synced 2024-12-25 08:39:09 -05:00
a526cff0a9
This PR adds a test reporter for the [Test Anything Protocol](https://testanything.org). It makes the following implementation decisions: - No TODO pragma, as there is no such marker in `Deno.test` - SKIP pragma for `ignore`d tests - Test steps are treated as TAP14 subtests - Support for this in consumers seems spotty - Some consumers will incorrectly interpret these markers, resulting in unexpected output - Considering the lack of support, and to avoid implementation complexity, subtests are at most one level deep (all test steps are in the same subtest) - To accommodate consumers that use comments to indicate test-suites (unspecced) - The test module path is output as a comment - This is disabled for `--parallel` testing - Failure diagnostics are output as JSON, which is also valid YAML - The structure is not specified, so the format roughly follows the spec example: ``` --- message: "Failed with error 'hostname peebles.example.com not found'" severity: fail found: hostname: 'peebles.example.com' address: ~ wanted: hostname: 'peebles.example.com' address: '85.193.201.85' at: file: test/dns-resolve.c line: 142 ... ```
43 lines
2.1 KiB
Text
43 lines
2.1 KiB
Text
TAP version 14
|
|
# ./test/steps/failing_steps.ts
|
|
# Subtest: nested failure
|
|
not ok 1 - inner 1
|
|
---
|
|
{"message":"Error: Failed.\n throw new Error(\"Failed.\");\n ^\n at [WILDCARD]/failing_steps.ts:[WILDCARD]\n[WILDCARD]","severity":"fail","at":{"file":"./test/steps/failing_steps.ts","line":[WILDCARD]}}
|
|
...
|
|
ok 2 - inner 2
|
|
not ok 3 - step 1
|
|
---
|
|
{"message":"1 test step failed.","severity":"fail","at":{"file":"./test/steps/failing_steps.ts","line":[WILDCARD]}}
|
|
...
|
|
1..3
|
|
not ok 1 - nested failure
|
|
---
|
|
{"message":"1 test step failed.","severity":"fail","at":{"file":"./test/steps/failing_steps.ts","line":[WILDCARD]}}
|
|
...
|
|
# Subtest: multiple test step failures
|
|
not ok 1 - step 1
|
|
---
|
|
{"message":"Error: Fail.\n throw new Error(\"Fail.\");\n ^\n at [WILDCARD]/failing_steps.ts:[WILDCARD]\n[WILDCARD]","severity":"fail","at":{"file":"./test/steps/failing_steps.ts","line":[WILDCARD]}}
|
|
...
|
|
not ok 2 - step 2
|
|
---
|
|
{"message":"Error: Fail.\n await t.step(\"step 2\", () => Promise.reject(new Error(\"Fail.\")));\n ^\n at [WILDCARD]/failing_steps.ts:[WILDCARD]\n[WILDCARD]","severity":"fail","at":{"file":"./test/steps/failing_steps.ts","line":[WILDCARD]}}
|
|
...
|
|
1..2
|
|
not ok 2 - multiple test step failures
|
|
---
|
|
{"message":"2 test steps failed.","severity":"fail","at":{"file":"./test/steps/failing_steps.ts","line":[WILDCARD]}}
|
|
...
|
|
# Subtest: failing step in failing test
|
|
not ok 1 - step 1
|
|
---
|
|
{"message":"Error: Fail.\n throw new Error(\"Fail.\");\n ^\n at [WILDCARD]/failing_steps.ts:[WILDCARD]\n[WILDCARD]","severity":"fail","at":{"file":"./test/steps/failing_steps.ts","line":[WILDCARD]}}
|
|
...
|
|
1..1
|
|
not ok 3 - failing step in failing test
|
|
---
|
|
{"message":"Error: Fail test.\n throw new Error(\"Fail test.\");\n ^\n at [WILDCARD]/failing_steps.ts:[WILDCARD]","severity":"fail","at":{"file":"./test/steps/failing_steps.ts","line":[WILDCARD]}}
|
|
...
|
|
1..3
|
|
error: Test failed
|