1
0
Fork 0
mirror of https://github.com/denoland/deno.git synced 2024-12-12 18:42:18 -05:00
Commit graph

89 commits

Author SHA1 Message Date
David Sherret
826e42a5b5
fix: improved support for cjs and cts modules (#26558)
* cts support
* better cjs/cts type checking
* deno compile cjs/cts support
* More efficient detect cjs (going towards stabilization)
* Determination of whether .js, .ts, .jsx, or .tsx is cjs or esm is only
done after loading
* Support `import x = require(...);`

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-11-01 12:27:00 -04:00
Leo Kettmeir
a1d0a427e8
feat: default to TS for file extension and support ext flag in more scenarios (#25472)
Closes #11220
Currently does lint, fmt, and repl
2024-09-18 21:15:13 +02:00
Yusuke Tanaka
d5c00ef50e
feat(cli): evaluate code snippets in JSDoc and markdown (#25220)
This commit lets `deno test --doc` command actually evaluate code snippets in
JSDoc and markdown files.

## How it works

1. Extract code snippets from JSDoc or code fences
2. Convert them into pseudo files by wrapping them in `Deno.test(...)`
3. Register the pseudo files as in-memory files
4. Run type-check and evaluation

We apply some magic at the step 2 - let's say we have the following file named
`mod.ts` as an input:

````ts
/**
 * ```ts
 * import { assertEquals } from "jsr:@std/assert/equals";
 *
 * assertEquals(add(1, 2), 3);
 * ```
 */
export function add(a: number, b: number) {
  return a + b;
}
````

This is virtually transformed into:

```ts
import { assertEquals } from "jsr:@std/assert/equals";
import { add } from "files:///path/to/mod.ts";

Deno.test("mod.ts$2-7.ts", async () => {
  assertEquals(add(1, 2), 3);
});
```

Note that a new import statement is inserted here to make `add` function
available. In a nutshell, all items exported from `mod.ts` become available in
the generated pseudo file with this automatic import insertion.

The intention behind this design is that, from library user's standpoint, it
should be very obvious that this `add` function is what this example code is
attached to. Also, if there is an explicit import statement like
`import { add } from "./mod.ts"`, this import path `./mod.ts` is not helpful for
doc readers because they will need to import it in a different way.

The automatic import insertion has some edge cases, in particular where there is
a local variable in a snippet with the same name as one of the exported items.
This case is addressed by employing swc's scope analysis (see test cases for
more details).

## "type-checking only" mode stays around

This change will likely impact a lot of existing doc tests in the ecosystem
because some doc tests rely on the fact that they are not evaluated - some cause
side effects if executed, some throw errors at runtime although they do pass the
type check, etc. To help those tests gradually transition to the ones runnable
with the new `deno test --doc`, we will keep providing the ability to run
type-checking only via `deno check --doc`. Additionally there is a `--doc-only`
option added to the `check` subcommand too, which is useful when you want to
type-check on code snippets in markdown files, as normal `deno check` command
doesn't accept markdown.

## Demo

https://github.com/user-attachments/assets/47e9af73-d16e-472d-b09e-1853b9e8f5ce

---

Closes #4716
2024-09-17 21:35:48 -07:00
David Sherret
62e952559f
refactor(permissions): split up Descriptor into Allow, Deny, and Query (#25508)
This makes the permission system more versatile.
2024-09-16 21:39:37 +01:00
Nayeem Rahman
ad30703e8e
fix(lsp): encode url parts before parsing as uri (#25509) 2024-09-11 11:11:39 +01:00
Nayeem Rahman
f959297dcd
feat(lsp): unstable setting as list (#25552) 2024-09-11 00:20:03 +01:00
Nayeem Rahman
97d1635343
fix(lsp): panic on url_to_uri() (#25238) 2024-08-28 05:15:48 +01:00
Nayeem Rahman
2ab4afc6b8
refactor(lsp): changes for lsp_types 0.97.0 (#25169) 2024-08-24 01:21:21 +01:00
Hajime-san
19bcb40059
feat(cli/tools): add a subcommand --hide-stacktraces for test (#24095) 2024-08-20 01:27:36 +00:00
David Sherret
d3419f7d14
perf: update deno_ast to 0.41 (#24819)
Some perf gains in swc (I measured formatting and it was slightly
faster).

Includes:

* https://github.com/denoland/deno_graph/pull/508
* https://github.com/denoland/eszip/pull/193
2024-07-31 18:31:03 -04:00
David Sherret
9114a2df69
fix(upgrade): do not error if config in cwd invalid (#24689)
```
> deno upgrade
error: Unsupported lockfile version 'invalid'. Try upgrading Deno or recreating the lockfile.
V:\scratch
> V:\deno\target\debug\deno upgrade
Looking up latest version
Local deno version 1.45.3 is the most recent release
```

Closes #24517
Closes #20729
2024-07-23 19:00:48 -04:00
Satya Rohith
0f48313565
chore: upgrade to rust 1.79 (#24207) 2024-06-14 17:10:57 +05:30
David Sherret
386d5c8310
refactor: remove PermissionsContainer in deno_runtime (#24119)
Also removes permissions being passed in for node resolution. It was
completely useless because we only checked it for reading package.json
files, but Deno reading package.json files for resolution is perfectly
fine.

My guess is this is also a perf improvement because Deno is doing less
work.
2024-06-06 23:37:53 -04:00
David Sherret
7ed90a20d0
fix: better handling of npm resolution occurring on workers (#24094)
Closes https://github.com/denoland/deno/issues/24063
2024-06-05 17:04:16 +02:00
Matt Mastracci
596a2996cf
feat(cli): Add slow test warning (#23874)
By default, uses a 60 second timeout, backing off 2x each time (can be
overridden using the hidden `DENO_SLOW_TEST_TIMEOUT` which we implement
only really for spec testing.

```
Deno.test(async function test() {
  await new Promise(r => setTimeout(r, 130_000));
});
```

```
$ target/debug/deno test /tmp/test_slow.ts 
Check file:///tmp/test_slow.ts
running 1 test from ../../../../../../tmp/test_slow.ts
test ...'test' is running very slowly (1m0s)
'test' is running very slowly (2m0s)
 ok (2m10s)

ok | 1 passed | 0 failed (2m10s)
```

---------

Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-05-22 08:08:27 -06:00
Divy Srivastava
88983fb3eb
fix(node): seperate worker module cache (#23634)
Construct a new module graph container for workers instead of sharing it
with the main worker.

Fixes #17248
Fixes #23461

---------

Co-authored-by: David Sherret <dsherret@gmail.com>
2024-05-16 07:09:35 +00:00
David Sherret
2dcbef2abb
fix(compile): relative permissions should be retained as relative (#23719)
Closes #23715
2024-05-06 19:21:58 -04:00
Nayeem Rahman
79e6751cf7
perf(lsp): only store parsed sources for open documents (#23454) 2024-04-20 02:00:03 +01:00
Nayeem Rahman
7446b0163d
perf(lsp): don't keep remote module ast's in memory (#23230) 2024-04-04 23:39:17 +01:00
Matt Mastracci
86cdf37033
perf(cli): use args_os (#23039)
Extracted from #22718
2024-03-22 14:03:56 -07:00
David Sherret
ad6b00a2bf
chore: enable clippy unused_async rule (#22834) 2024-03-11 23:48:00 -04:00
Matt Mastracci
0fdb33c3aa
fix(cli): limit test parallelism on Windows to avoid pipe error (#22776)
One last attempt to fix the parallelism issue on Windows.
2024-03-07 10:04:39 -07:00
Matt Mastracci
b6e44f91ad
fix(cli): ensure that pre- and post-test output is flushed at the appropriate times (#22611)
Some `deno_std` tests were failing to print output that was resolved
after the last test finished. In addition, output printed before tests
began would sometimes appear above the "running X tests ..." line, and
sometimes below it depending on timing.

We now guarantee that all output is flushed before and after tests run,
making the output consistent.

Pre-test and post-test output are captured in `------ pre-test output
------` and `------ post-test output ------` blocks to differentiate
them from the regular output blocks.

Here's an example of a test (that is much noisier than normal, but an
example of what the output will look like):

```
Check ./load_unload.ts
------- pre-test output -------
load
----- output end -----
running 1 test from ./load_unload.ts
test ...
------- output -------
test
----- output end -----
test ... ok ([WILDCARD])
------- post-test output -------
unload
----- output end -----
```
2024-02-28 22:12:21 +00:00
Matt Mastracci
a3f982c1d5
chore(cli): rename --trace-ops to --trace-leaks (#22598)
As we add tracing to more types of runtime activity, `--trace-ops` is
less useful of a name. `--trace-leaks` better reflects that this feature
traces both ops and timers, and will eventually trace resource opening
as well.

This keeps `--trace-ops` as an alias for `--trace-leaks`, but prints a
warning to the console suggesting migration to `--trace-leaks`.

One test continues to use `--trace-ops` to test the deprecation warning.

---------

Signed-off-by: Matt Mastracci <matthew@mastracci.com>
2024-02-28 09:12:43 -07:00
Matt Mastracci
96cfe82664
perf(cli): reduce overhead in test registration (#22552)
- Removes the origin call, since all origins are the same for an isolate
(ie: the main module)
- Collects the `TestDescription`s and sends them all at the same time
inside of an Arc, allowing us to (later on) re-use these instead of
cloning.

Needs a follow-up pass to remove all the cloning, but that's a thread
that is pretty long to pull

---------

Signed-off-by: Matt Mastracci <matthew@mastracci.com>
2024-02-27 20:30:17 -07:00
Matt Mastracci
5193834cf2
refactor(cli): clean up test runner channels (#22422)
Gets us closer to solving #20707.

Rewrites the `TestEventSender`:

- Allow for explicit creation of multiple streams. This will allow for
one-std{out,err}-per-worker
- All test events are received along with a worker ID, allowing for
eventual, proper parallel threading of test events.
 
In theory this should open up proper interleaving of test output,
however that is left for a future PR.

I had some plans for a better performing synchronization primitive, but
the inter-thread communication is tricky. This does, however, speed up
the processing of large numbers of tests 15-25% (possibly even more on
100,000+).

Before

```
ok | 1000 passed | 0 failed (32ms)
ok | 10000 passed | 0 failed (276ms)
```

After

```
ok | 1000 passed | 0 failed (25ms)
ok | 10000 passed | 0 failed (230ms)
```
2024-02-23 11:11:15 -07:00
David Sherret
e5e2c45998
fix: upgrade to deno_ast 0.33 (#22341)
* Uses diagnostics from deno_ast
* Real fix for https://github.com/denoland/deno/pull/22310
* Moves `deno lint --json` code here
* Upgrades swc
   
 Closes #22117
 Closes #22109
 Closes #21927
 Closes #20993
2024-02-09 01:40:26 +00:00
Matt Mastracci
aecad7f353
refactor(cli): Add TestFailureDescription (#22267)
Extract zero-risk changes from #22226
2024-02-05 17:27:17 +00:00
Bartek Iwańczuk
7281775381
Revert "refactor(cli): use new sanitizer for resources (#22125)" (#22153) 2024-01-27 20:37:36 +01:00
Matt Mastracci
84fb2ad71b
refactor(cli): use new sanitizer for resources (#22125)
Step 1 of the Rustification of sanitizers, which unblocks the faster
timers.

This replaces the resource sanitizer with a Rust one, using the new APIs
in deno_core.
2024-01-26 17:24:16 -05:00
林炳权
96b581bdd2
chore: update to Rust 1.75 (#21731) 2024-01-01 23:22:48 +01:00
David Sherret
7e72f3af61
chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
Bartek Iwańczuk
fe90ba650d
refactor(lsp): log names (#21413)
This commit changes LSP log names by prefixing them, we now have these
prefixes:
- `lsp.*` - requests coming from the client
- `tsc.request.*` - requests coming from clients that are routed to TSC
- `tsc.op.*` - ops called by the TS host
- `tsc.host.*` - requests that call JavaScript runtime that runs
TypeScript compiler host

Additionall `Performance::mark` was split into `Performance::mark` and 
`Performance::mark_with_args` to reduce verbosity of code and logs.
2023-12-01 03:54:59 +01:00
David Sherret
8f065a60e7
fix: improved using declaration support (#20959)
Upgrades to deno_ast 0.30.
2023-10-24 09:37:02 -04:00
Nayeem Rahman
551a081450
refactor(test): support custom writer in PrettyTestReporter (#20783) 2023-10-05 11:25:15 +01:00
Nayeem Rahman
f75a17521d
fix(lsp): respect configured exclusions for testing APIs (#20427)
LSP testing APIs now obey the various file inclusion settings:
- Modules shown in the text explorer now respect the `exclude`,
`test.exclude` and `test.include` fields in `deno.json`, as well as
`deno.enablePaths` in VSCode settings.
- Modules with testing code lens now respect the `"exclude"`,
`test.exclude` and `test.include` fields in `deno.json`. Code lens
already respects `deno.enablePaths`.
2023-09-09 19:37:01 +01:00
Bartek Iwańczuk
147c845c95
feat(test): Add Deno.test.ignore and Deno.test.only (#20365)
Closes https://github.com/denoland/deno/issues/17106
2023-09-06 14:17:33 +02:00
Nayeem Rahman
d28384c3de
refactor(lsp): store test definitions in adjacency list (#20330)
Previously:
```rust
pub struct TestDefinition {
  pub id: String,
  pub name: String,
  pub range: SourceRange,
  pub steps: Vec<TestDefinition>,
}

pub struct TestDefinitions {
  pub discovered: Vec<TestDefinition>,
  pub injected: Vec<lsp_custom::TestData>,
  pub script_version: String,
}
```
Now:
```rust
pub struct TestDefinition {
  pub id: String,
  pub name: String,
  pub range: Option<Range>,
  pub is_dynamic: bool, // True for 'injected' module, not statically detected but added at runtime.
  pub parent_id: Option<String>,
  pub step_ids: HashSet<String>,
}

pub struct TestModule {
  pub specifier: ModuleSpecifier,
  pub script_version: String,
  pub defs: HashMap<String, TestDefinition>,
}
```

Storing the test tree as a literal tree diminishes the value of IDs,
even though vscode stores them that way. This makes all data easily
accessible from `TestModule`. It unifies the interface between
'discovered' and 'injected' tests. This unblocks some enhancements wrt
syncing tests between the LSP and extension, such as this TODO:
61f08d5a71/client/src/testing.ts (L251-L259)
and https://github.com/denoland/vscode_deno/issues/900. We should also
get more flexibility overall.

`TestCollector` is cleaned up, now stores a `&mut TestModule` directly
and registers tests as it comes across them with
`TestModule::register()`. This method ensures sanity in the redundant
data from having both of `TestDefinition::{parent_id,step_ids}`.

All of the messy conversions between `TestDescription`,
`LspTestDescription`, `TestDefinition`, `TestData` and `TestIdentifier`
are cleaned up. They shouldn't have been using `impl From` and now the
full list of tests is available to their implementations.
2023-08-30 16:31:31 +01:00
Nayeem Rahman
c31c93ce70
fix(lsp): delete test modules with all tests deleted (#20321)
Fixes https://github.com/denoland/vscode_deno/issues/899.
2023-08-29 09:43:11 +01:00
Nayeem Rahman
e1fe31508c
fix(lsp/testing): use full ancestry to compute static id of step (#20297)
Fixes https://github.com/denoland/vscode_deno/issues/656.

Test steps were ID'd by a checksum of `[origin, level, step_name]` which
is incorrect. Now it's `[origin, ...ancestor_names, step_name]`.
2023-08-27 11:16:09 +02:00
Nayeem Rahman
907d9bb4d7
fix(lsp): test explorer panic on step result (#20289)
Fixes https://github.com/denoland/vscode_deno/issues/843.

Prevents step results from being reported twice. Refactors
`LspTestReporter` to use a complete `(test_id, descriptor)` map instead
of a brittle `LspTestReporter::stack`.
2023-08-25 23:32:22 +02:00
Nayeem Rahman
9e31f9452e
fix(lsp/testing): don't queue modules without tests (#20277)
Fixes https://github.com/denoland/vscode_deno/issues/890.
2023-08-25 11:12:11 +02:00
Matt Mastracci
b1ce2e4167
fix(ext/web): add stream tests to detect v8slice split bug (#20253)
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-08-23 17:03:05 -06:00
Bartek Iwańczuk
029bdf0cd5
feat(cli): Add dot test reporter (#19804)
This commit adds a "dot" reporter to "deno test" subcommand,
that can be activated using "--dot" flag.

It provides a concise output using:
- "." for passing test
- "," for ignored test
- "!" for failing test

User output is silenced and not printed to the console.

In non-TTY environments each result is printed on a separate line.
2023-08-02 18:38:10 +02:00
Matt Mastracci
9845361153
refactor(core): bake single-thread assumptions into spawn/spawn_blocking (#19056)
Partially supersedes #19016.

This migrates `spawn` and `spawn_blocking` to `deno_core`, and removes
the requirement for `spawn` tasks to be `Send` given our single-threaded
executor.

While we don't need to technically do anything w/`spawn_blocking`, this
allows us to have a single `JoinHandle` type that works for both cases,
and allows us to more easily experiment with alternative
`spawn_blocking` implementations that do not require tokio (ie: rayon).

Async ops (+~35%):

Before: 

```
time 1310 ms rate 763358
time 1267 ms rate 789265
time 1259 ms rate 794281
time 1266 ms rate 789889
```

After:

```
time 956 ms rate 1046025
time 954 ms rate 1048218
time 924 ms rate 1082251
time 920 ms rate 1086956
```

HTTP serve (+~4.4%):

Before:

```
Running 10s test @ http://localhost:4500
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    68.78us   19.77us   1.43ms   86.84%
    Req/Sec    68.78k     5.00k   73.84k    91.58%
  1381833 requests in 10.10s, 167.36MB read
Requests/sec: 136823.29
Transfer/sec:     16.57MB
```

After:

```
Running 10s test @ http://localhost:4500
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    63.12us   17.43us   1.11ms   85.13%
    Req/Sec    71.82k     3.71k   77.02k    79.21%
  1443195 requests in 10.10s, 174.79MB read
Requests/sec: 142921.99
Transfer/sec:     17.31MB
```

Suggested-By: alice@ryhl.io
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-05-14 15:40:01 -06:00
David Sherret
9efed4c7a3
refactor(cli): remove ProcState - add CliFactory (#18900)
This removes `ProcState` and replaces it with a new `CliFactory` which
initializes our "service structs" on demand. This isn't a performance
improvement at the moment for `deno run`, but might unlock performance
improvements in the future.
2023-05-01 14:35:23 -04:00
David Sherret
9a9473533e
refactor(cli): remove Clone on ProcState (#18874)
Slowly phasing this out.
2023-04-30 16:51:31 -04:00
David Sherret
742cc3111c
refactor(cli): extract out ProcState from CliMainWorker (#18867) 2023-04-27 10:05:20 -04:00
Nayeem Rahman
3d8a4d3b81
feat(cli): don't check permissions for statically analyzable dynamic imports (#18713)
Closes #17697
Closes #17658
2023-04-26 16:23:28 -04:00
Nayeem Rahman
6e8618ae0f
refactor(cli): move runTests() and runBenchmarks() to rust (#18563)
Stores the test/bench functions in rust op state during registration.
The functions are wrapped in JS first so that they return a directly
convertible `TestResult`/`BenchResult`. Test steps are still mostly
handled in JS since they are pretty much invoked by the user. Allows
removing a bunch of infrastructure for communicating between JS and
rust. Allows using rust utilities for things like shuffling tests
(`Vec::shuffle`). We can progressively move op and resource sanitization
to rust as well.

Fixes #17122.
Fixes #17312.
2023-04-13 19:43:23 +02:00