Currently deno eagerly caches all npm packages in the workspace's npm
resolution. So, for instance, running a file `foo.ts` that imports
`npm:chalk` will also install all dependencies listed in `package.json`
and all `npm` dependencies listed in the lockfile.
This PR refactors things to give more control over when and what npm
packages are automatically cached while building the module graph.
After this PR, by default the current behavior is unchanged _except_ for
`deno install --entrypoint`, which will only cache npm packages used by
the given entrypoint. For the other subcommands, this behavior can be
enabled with `--unstable-npm-lazy-caching`
Fixes #25782.
---------
Signed-off-by: Nathan Whitaker <17734409+nathanwhit@users.noreply.github.com>
Co-authored-by: Luca Casonato <hello@lcas.dev>
Ensures a dynamic import in a CJS file will consider the referrer as an import for node resolution.
Also adds fixes (adds) support for `"resolution-mode"` in TypeScript.
This will respect `"type": "commonjs"` in a package.json to determine if
`.js`/`.jsx`/`.ts`/.tsx` files are CJS or ESM. If the file is found to
be ESM it will be loaded as ESM though.
* cts support
* better cjs/cts type checking
* deno compile cjs/cts support
* More efficient detect cjs (going towards stabilization)
* Determination of whether .js, .ts, .jsx, or .tsx is cjs or esm is only
done after loading
* Support `import x = require(...);`
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit fixes memory leak described in
https://github.com/denoland/deno/issues/24380.
This is done by upgrading following crates:
- deno_ast
- deno_graph
- eszip
- dprint-plugin-typescript
- deno_lint
- deno_doc
- deno_emit
`TestEventSender` should not be Clone so we don't end up with multiple
copies of the same writer FD. This is probably not the cause of the test
channel lockups, but it's a lot easier to reason about.
This patch gets JUnit reporter to output more detailed information for
test steps (subtests).
## Issue with previous implementation
In the previous implementation, the test hierarchy was represented using
several XML tags like the following:
- `<testsuites>` corresponds to the entire test (one execution of `deno
test` has exactly one `<testsuites>` tag)
- `<testsuite>` corresponds to one file, such as `main_test.ts`
- `<testcase>` corresponds to one `Deno.test(...)`
- `<property>` corresponds to one `t.step(...)`
This structure describes the test layers but one problem is that
`<property>` tag is used for any use cases so some tools that can ingest
a JUnit XML file might not be able to interpret `<property>` as
subtests.
## How other tools address it
Some of the testing frameworks in the ecosystem address this issue by
fitting subtests into the `<testcase>` layer. For instance, take a look
at the following Go test file:
```go
package main_test
import "testing"
func TestMain(t *testing.T) {
t.Run("child 1", func(t *testing.T) {
// OK
})
t.Run("child 2", func(t *testing.T) {
// Error
t.Fatal("error")
})
}
```
Running [gotestsum], we can get the output like this:
```xml
<?xml version="1.0" encoding="UTF-8"?>
<testsuites tests="3" failures="2" errors="0" time="1.013694">
<testsuite tests="3" failures="2" time="0.510000" name="example/gosumtest" timestamp="2024-03-11T12:26:39+09:00">
<properties>
<property name="go.version" value="go1.22.1 darwin/arm64"></property>
</properties>
<testcase classname="example/gosumtest" name="TestMain/child_2" time="0.000000">
<failure message="Failed" type="">=== RUN TestMain/child_2
 main_test.go:12: error
--- FAIL: TestMain/child_2 (0.00s)
</failure>
</testcase>
<testcase classname="example/gosumtest" name="TestMain" time="0.000000">
<failure message="Failed" type="">=== RUN TestMain
--- FAIL: TestMain (0.00s)
</failure>
</testcase>
<testcase classname="example/gosumtest" name="TestMain/child_1" time="0.000000"></testcase>
</testsuite>
</testsuites>
```
This output shows that nested test cases are squashed into the
`<testcase>` layer by treating them as the same layer as their parent,
`TestMain`. We can still distinguish nested ones by their `name`
attributes that look like `TestMain/<subtest_name>`.
As described in #22795, [vitest] solves the issue in the same way as
[gotestsum].
One downside of this would be that one test failure that happens in a
nested test case will end up being counted multiple times, because not
only the subtest but also its wrapping container(s) are considered to be
failures. In fact, in the [gotestsum] output above, `TestMain/child_2`
failed (which is totally expected) while its parent, `TestMain`, was
also counted as failure. As
https://github.com/denoland/deno/pull/20273#discussion_r1307558757
pointed out, there is a test runner that offers flexibility to prevent
this, but I personally don't think the "duplicate failure count" issue
is a big deal.
## How to fix the issue in this patch
This patch fixes the issue with the same approach as [gotestsum] and
[vitest].
More specifically, nested test cases are put into the `<testcase>` level
and their names are now represented as squashed test names concatenated
by `>` (e.g. `parent 2 > child 1 > grandchild 1`). This change also
allows us to put a detailed error message as `<failure>` tag within the
`<testcase>` tag, which should be handled nicely by third-party tools
supporting JUnit XML.
## Extra fix
Also, file paths embedded into XML outputs are changed from absolute
path to relative path, which is helpful when running the test suites in
several different environments like CI.
Resolves #22795
[gotestsum]: https://github.com/gotestyourself/gotestsum
[vitest]: https://vitest.dev/
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Gets us closer to solving #20707.
Rewrites the `TestEventSender`:
- Allow for explicit creation of multiple streams. This will allow for
one-std{out,err}-per-worker
- All test events are received along with a worker ID, allowing for
eventual, proper parallel threading of test events.
In theory this should open up proper interleaving of test output,
however that is left for a future PR.
I had some plans for a better performing synchronization primitive, but
the inter-thread communication is tricky. This does, however, speed up
the processing of large numbers of tests 15-25% (possibly even more on
100,000+).
Before
```
ok | 1000 passed | 0 failed (32ms)
ok | 10000 passed | 0 failed (276ms)
```
After
```
ok | 1000 passed | 0 failed (25ms)
ok | 10000 passed | 0 failed (230ms)
```
This commit adds support for [TC39 Decorator
Proposal](https://github.com/tc39/proposal-decorators).
These decorators are only available in transpiled sources - ie.
non-JavaScript files (because of lack of support in V8).
This entails that "experimental TypeScript decorators" are not available
by default
and require to be configured, with a configuration like this:
```
{
"compilerOptions": {
"experimentalDecorators": true
}
}
```
Closes https://github.com/denoland/deno/issues/19160
---------
Signed-off-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: crowlkats <crowlkats@toaxl.com>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Adds an `--unstable-sloppy-imports` flag which supports the
following for `file:` specifiers:
* Allows writing `./mod` in a specifier to do extension probing.
- ex. `import { Example } from "./example"` instead of `import { Example
} from "./example.ts"`
* Allows writing `./routes` to do directory extension probing for files
like `./routes/index.ts`
* Allows writing `./mod.js` for *mod.ts* files.
This functionality is **NOT RECOMMENDED** for general use with Deno:
1. It's not as optimal for perf:
https://marvinh.dev/blog/speeding-up-javascript-ecosystem-part-2/
1. It makes tooling in the ecosystem more complex in order to have to
understand this.
1. The "Deno way" is to be explicit about what you're doing. It's better
in the long run.
1. It doesn't work if published to the Deno registry because doing stuff
like extension probing with remote specifiers would be incredibly slow.
This is instead only recommended to help with migrating existing
projects to Deno. For example, it's very useful for getting CJS projects
written with import/export declaration working in Deno without modifying
module specifiers and for supporting TS ESM projects written with
`./mod.js` specifiers.
This feature will output warnings to guide the user towards correcting
their specifiers. Additionally, quick fixes are provided in the LSP to
update these specifiers:
This commit moves all Chrome Devtools Protocol messages to `cli/cdp.rs`
and refactors all places using these types to pull them from a common
place.
No functional changes.
This makes `CliNpmResolver` a trait. The terminology used is:
- **managed** - Deno manages the node_modules folder and does an
auto-install (ex. `ManagedCliNpmResolver`)
- **byonm** - "Bring your own node_modules" (ex. `ByonmCliNpmResolver`,
which is in this PR, but unimplemented at the moment)
Part of #18967
This commit adds "deno jupyter" subcommand which
provides a Deno kernel for Jupyter notebooks.
The implementation is mostly based on Deno's REPL and
reuses large parts of it (though there's some clean up that
needs to happen in follow up PRs). Not all functionality of
Jupyter kernel is implemented and some message type
are still not implemented (eg. "inspect_request") but
the kernel is fully working and provides all the capatibilities
that the Deno REPL has; including TypeScript transpilation
and npm packages support.
Closes https://github.com/denoland/deno/issues/13016
---------
Co-authored-by: Adam Powers <apowers@ato.ms>
Co-authored-by: Kyle Kelley <rgbkrk@gmail.com>
To fix bugs around detection of when node emulation is required, we will
just eagerly initialize it. The improvements we make to reduce the
impact of the startup time:
- [x] Process stdin/stdout/stderr are lazily created
- [x] node.js global proxy no longer allocates on each access check
- [x] Process checks for `beforeExit` listeners before doing expensive
shutdown work
- [x] Process should avoid adding global event handlers until listeners
are added
Benchmarking this PR (`89de7e1ff`) vs main (`41cad2179`)
```
12:36 $ third_party/prebuilt/mac/hyperfine --warmup 100 -S none './deno-41cad2179 run ./empty.js' './deno-89de7e1ff run ./empty.js'
Benchmark 1: ./deno-41cad2179 run ./empty.js
Time (mean ± σ): 24.3 ms ± 1.6 ms [User: 16.2 ms, System: 6.0 ms]
Range (min … max): 21.1 ms … 29.1 ms 115 runs
Benchmark 2: ./deno-89de7e1ff run ./empty.js
Time (mean ± σ): 24.0 ms ± 1.4 ms [User: 16.3 ms, System: 5.6 ms]
Range (min … max): 21.3 ms … 28.6 ms 126 runs
```
Fixes https://github.com/denoland/deno/issues/20142
Fixes https://github.com/denoland/deno/issues/15826
Fixes https://github.com/denoland/deno/issues/20028
Fixes regression from #18878 where `Promise.reject()`,
`Promise.reject(undefined)` and `reportError(undefined)` panic in the
REPL.
Fixes `throw undefined` printing `Uncaught Unknown exception` instead of
`Uncaught undefined`.
Fixes #8858.
Fixes #8869.
```
$ target/debug/deno
Deno 1.32.5
exit using ctrl+d, ctrl+c, or close()
REPL is running with all permissions allowed.
To specify permissions, run `deno repl` with allow flags.
> Promise.reject(new Error("bar"));
Promise { <rejected> Error: bar
at <anonymous>:2:16 }
Uncaught (in promise) Error: bar
at <anonymous>:2:16
> reportError(new Error("baz"));
undefined
Uncaught Error: baz
at <anonymous>:2:13
>
1. Fixes a cosmetic issue in the repl where it would display lsp warning
messages.
2. Lazily loads dependencies from the package.json on use.
3. Supports using bare specifiers from package.json in the REPL.
Closes #17929
Closes #18494
Moving some code around in `ext/node` is it's a bit better well defined
and makes it possible for others to embed it.
I expect to see no difference in startup perf with this change.
These functions don't need to be async, as they are only calling
synchronous JavaScript code. As a follow up, all 3 functions
should be merge together - this will reduce roundtrips for
calling V8 from Rust, which is somewhat expensive
