1
0
Fork 0
mirror of https://github.com/denoland/deno.git synced 2024-12-22 15:24:46 -05:00
denoland-deno/tests/specs/test
Yusuke Tanaka d5c00ef50e
feat(cli): evaluate code snippets in JSDoc and markdown (#25220)
This commit lets `deno test --doc` command actually evaluate code snippets in
JSDoc and markdown files.

## How it works

1. Extract code snippets from JSDoc or code fences
2. Convert them into pseudo files by wrapping them in `Deno.test(...)`
3. Register the pseudo files as in-memory files
4. Run type-check and evaluation

We apply some magic at the step 2 - let's say we have the following file named
`mod.ts` as an input:

````ts
/**
 * ```ts
 * import { assertEquals } from "jsr:@std/assert/equals";
 *
 * assertEquals(add(1, 2), 3);
 * ```
 */
export function add(a: number, b: number) {
  return a + b;
}
````

This is virtually transformed into:

```ts
import { assertEquals } from "jsr:@std/assert/equals";
import { add } from "files:///path/to/mod.ts";

Deno.test("mod.ts$2-7.ts", async () => {
  assertEquals(add(1, 2), 3);
});
```

Note that a new import statement is inserted here to make `add` function
available. In a nutshell, all items exported from `mod.ts` become available in
the generated pseudo file with this automatic import insertion.

The intention behind this design is that, from library user's standpoint, it
should be very obvious that this `add` function is what this example code is
attached to. Also, if there is an explicit import statement like
`import { add } from "./mod.ts"`, this import path `./mod.ts` is not helpful for
doc readers because they will need to import it in a different way.

The automatic import insertion has some edge cases, in particular where there is
a local variable in a snippet with the same name as one of the exported items.
This case is addressed by employing swc's scope analysis (see test cases for
more details).

## "type-checking only" mode stays around

This change will likely impact a lot of existing doc tests in the ecosystem
because some doc tests rely on the fact that they are not evaluated - some cause
side effects if executed, some throw errors at runtime although they do pass the
type check, etc. To help those tests gradually transition to the ones runnable
with the new `deno test --doc`, we will keep providing the ability to run
type-checking only via `deno check --doc`. Additionally there is a `--doc-only`
option added to the `check` subcommand too, which is useful when you want to
type-check on code snippets in markdown files, as normal `deno check` command
doesn't accept markdown.

## Demo

https://github.com/user-attachments/assets/47e9af73-d16e-472d-b09e-1853b9e8f5ce

---

Closes #4716
2024-09-17 21:35:48 -07:00
..
aggregate_error chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
allow_all chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
allow_none chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
before_unload_prevent_default chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
captured_output feat: warn when using --allow-run with no allow list (#25215) 2024-09-16 23:08:02 +00:00
check_local_by_default chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
check_local_by_default2 chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
clean_flag chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
clear_timeout chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
collect chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
doc feat(cli): evaluate code snippets in JSDoc and markdown (#25220) 2024-09-17 21:35:48 -07:00
doc_duplicate_identifier feat(cli): evaluate code snippets in JSDoc and markdown (#25220) 2024-09-17 21:35:48 -07:00
doc_failure feat(cli): evaluate code snippets in JSDoc and markdown (#25220) 2024-09-17 21:35:48 -07:00
doc_only feat(cli): evaluate code snippets in JSDoc and markdown (#25220) 2024-09-17 21:35:48 -07:00
doc_permission_respected feat(cli): evaluate code snippets in JSDoc and markdown (#25220) 2024-09-17 21:35:48 -07:00
doc_success feat(cli): evaluate code snippets in JSDoc and markdown (#25220) 2024-09-17 21:35:48 -07:00
doc_ts_declare_global feat(cli): evaluate code snippets in JSDoc and markdown (#25220) 2024-09-17 21:35:48 -07:00
doc_ts_namespace_decl feat(cli): evaluate code snippets in JSDoc and markdown (#25220) 2024-09-17 21:35:48 -07:00
exit_code refactor(cli/js): align error messages (#25406) 2024-09-04 09:19:55 +02:00
exit_code2 refactor(cli/js): align error messages (#25406) 2024-09-04 09:19:55 +02:00
exit_code3 refactor(cli/js): align error messages (#25406) 2024-09-04 09:19:55 +02:00
exit_sanitizer chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
fail chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
fail_fast chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
fail_with_contain_unicode_filename chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
file_protocol chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
filter chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
finally_timeout chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
hide_empty_suites chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
hide_stacktraces feat(cli/tools): add a subcommand --hide-stacktraces for test (#24095) 2024-08-20 01:27:36 +00:00
ignore chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
ignore_persmissions chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
include_relative_pattern_dot_slash chore: migrate bench, publish, and more itests to spec tests (#23584) 2024-04-29 10:08:27 -04:00
interval chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
junit chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
junit_multiple_test_files chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
junit_nested chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
load_unload chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
markdown feat(cli): evaluate code snippets in JSDoc and markdown (#25220) 2024-09-17 21:35:48 -07:00
markdown_full_block_names feat(cli): evaluate code snippets in JSDoc and markdown (#25220) 2024-09-17 21:35:48 -07:00
markdown_ignore_html_comment feat(cli): evaluate code snippets in JSDoc and markdown (#25220) 2024-09-17 21:35:48 -07:00
markdown_windows feat(cli): evaluate code snippets in JSDoc and markdown (#25220) 2024-09-17 21:35:48 -07:00
meta chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
no_check chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
no_color chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
no_files BREAKING: remove --allow-none flag (#25337) 2024-09-02 22:06:27 +00:00
no_prompt_by_default chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
no_prompt_with_denied_perms chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
no_run chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
non_error_thrown chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
only chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
ops_sanitizer_closed_inside_started_before chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
ops_sanitizer_multiple_timeout_tests chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
ops_sanitizer_multiple_timeout_tests_no_trace chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
ops_sanitizer_nexttick chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
ops_sanitizer_tcp chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
ops_sanitizer_timeout_failure chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
ops_sanitizer_unstable chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
opt_out_top_level_exclude_via_test_unexclude chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
overloads chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
package_json_basic test: rewrite some ignored tests to spec tests (#25652) 2024-09-16 15:39:58 +02:00
package_json_basic_auto_install test: rewrite some ignored tests to spec tests (#25652) 2024-09-16 15:39:58 +02:00
parallel_flag chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
parallel_flag_with_env_variables chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
parallel_output chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
pass chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
quiet chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
recursive_permissions_pledge chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
report_error chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
resource_sanitizer chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
sanitizer_trace_ops_catch_error chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
sanitizer_with_error chore(specs): use jsonc for metadata file (#22946) 2024-03-15 17:27:52 +00:00
sanitizer_with_top_level_ops chore(specs): use jsonc for metadata file (#22946) 2024-03-15 17:27:52 +00:00
shuffle chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
shuffle_with_seed chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
slow_test feat(cli): Add slow test warning (#23874) 2024-05-22 08:08:27 -06:00
steps_dot_failing_steps chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
steps_dot_ignored_steps chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
steps_dot_passing_steps chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
steps_failing_steps chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
steps_ignored_steps chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
steps_invalid_usage chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
steps_output_within chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
steps_passing_steps chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
steps_tap_failing_steps chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
steps_tap_ignored_steps chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
steps_tap_passing_steps chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
test_and_bench fix: don't panic in test and bench if ops not available (#23055) 2024-03-24 16:16:45 -07:00
test_filtered_out_only chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
test_replace_timers chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
test_with_config chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
test_with_config2 chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
test_with_custom_jsx chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
test_with_glob_config chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
test_with_malformed_config chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
text chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
type_check_with_doc feat(cli): evaluate code snippets in JSDoc and markdown (#25220) 2024-09-17 21:35:48 -07:00
uncaught_errors chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
unhandled_rejection chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
unresolved_promise chore: deprecate test itests (#25512) 2024-09-16 19:38:40 +00:00
worker_large_output fix(cli): Identify and fix a test deadlock (#23411) 2024-04-16 15:14:59 -06:00
workspace feat: npm workspace and better Deno workspace support (#24334) 2024-07-04 00:54:33 +00:00