Extracting out more code from the CLI for reuse elsewhere (still more
work to do, but this is a start).
This is the code for extracting npm tarballs and saving information in
the npm cache in the global deno_dir.
Improving the breadth of collected data, and ensuring that the collected
data is more likely to be successfully reported.
- Use `log` crate in more places
- Hook up `log` crate to otel
- Switch to process-wide otel processors
- Handle places that use `process::exit`
Also adds a more robust testing framework, with a deterministic tracing
setting.
Refs: https://github.com/denoland/deno/issues/26852
This commit makes comments in code snippets in JSDoc or markdown
preserved when they are executed as tests. In particular, this is needed
to get TypeScript special comments such as `@ts-ignore` or
`@ts-expect-error` to work correctly.
Fixes #26728
Fixes #26116.
Handle the new error and treat is as lacking permission to make
symlinks, but also to make this more robust, just always fall back to
junctions no matter what the actual error is. Instead, warn if the error
isn't one we've handled, but go on to attempt creating the junction
* cts support
* better cjs/cts type checking
* deno compile cjs/cts support
* More efficient detect cjs (going towards stabilization)
* Determination of whether .js, .ts, .jsx, or .tsx is cjs or esm is only
done after loading
* Support `import x = require(...);`
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Fixes percent encodings showing up when installing scoped packages via
`deno add` or `deno install`. The issue is caused by us trying to map
back the package name from the resolved http url. This doesn't and has
never worked with private registries. The proper solution would be to
pass the original specifier into here, but that's a bit of a bigger
refactor.
So for now the quickest workaround is to replace `%2f` back to `/`.
Fixes https://github.com/denoland/deno/issues/26576
The --watch option should clear the screen scrollback buffer as well as
the screen itself.
On Ubuntu (22.04 Jammy) the 'clear' command generates
"\x1B[H\x1B[2J\x1B[3J"; that is:
- \E[H - cursor home
- \E[2J - clear entire screen
- \E[3J - clear entire screen & scrollback buffer.
By contrast, Deno defined CLEAR_SCREEN as "\x1B[2J\x1B[1;1H", which
fails to clear the scrollback buffer.
The "\E[H\E[2J\E[3J" sequence works on MacOS (Sonoma) (using printf);
I'm not able to test on Windows.
Closes https://github.com/denoland/deno/issues/26514
Fixes #26085.
Adds a basic retry utility with some defaults, starts off with a 100ms
wait, then 250ms, then 500ms
I've applied the retry in the http client, reusing an existing function,
so this also applies to retrying downloads of deno binaries in `upgrade`
and `compile`. I can make a separate function that doesn't retry so this
doesn't affect `upgrade` and `compile`, but it seemed desirable to have
retries there too, so I left it in.
This commit fixes the issue of import name conflict in case the named
export and default export have the same name by letting named export
take precedence over default export.
Fixes #26009
When using the `--unstable-detect-cjs` flag or adding `"unstable":
["detect-cjs"]` to a deno.json, it will make a JS file CJS if the
closest package.json contains `"type": "commonjs"` and the file is not
an ESM module (no TLA, no `import.meta`, no `import`/`export`).
This commit fixes issues with the pseudo test file generation logic,
namely:
- `export`s declared in snippets
- auto import insertion for `default export`
## Case 1: `export`s declared in snippets
In the previous implementation, `export`s declared in snippets were
moved to the top level of the module in the generated pseudo test file.
This is required because `export` must be at the top level.
This becomes a problem if such a `export` has a body, containing a
reference to a local variable. Suppose we extract this snippet from
JSDoc:
```ts
const logger = createLogger("my-awesome-module");
export function sum(a: number, b: number): number {
logger.debug("sum called");
return a + b;
}
```
This gets converted into the following invalid code (note that `export
function sum` is moved to the top level, but its body references
`logger` variable which can't be referenced from here):
```ts
export function sum(a: number, b: number): number {
logger.debug("sum called");
return a + b;
}
Deno.test("./base.ts$1-7.ts", async () => {
const logger = createLogger("my-awesome-module");
});
```
To resolve this issue, this commit adds a logic to remove the `export`
keyword, allowing the exported items to stay in the `Deno.test` block
scope, like so:
```ts
Deno.test("./base.ts$1-7.ts", async () => {
const logger = createLogger("my-awesome-module");
function sum(a: number, b: number): number {
logger.debug("sum called");
return a + b;
}
});
```
## Case 2: default export
Previously `default export foo` was not captured by the export
collector, so auto import insertion didn't work for this case. To put it
concretely, the following code snippet didn't work when run with `deno
test --doc` because `import foo from "file:///path/to/mod.ts"` didn't
get inserted automatically:
```ts
/**
* ```ts
* console.log(foo);
* ```
*
* @module
*/
const foo = 42;
export default foo;
```
This commit fixes this issue and the above example works fine.
---
Fixes #25718
This commit lets `deno test --doc` command actually evaluate code snippets in
JSDoc and markdown files.
## How it works
1. Extract code snippets from JSDoc or code fences
2. Convert them into pseudo files by wrapping them in `Deno.test(...)`
3. Register the pseudo files as in-memory files
4. Run type-check and evaluation
We apply some magic at the step 2 - let's say we have the following file named
`mod.ts` as an input:
````ts
/**
* ```ts
* import { assertEquals } from "jsr:@std/assert/equals";
*
* assertEquals(add(1, 2), 3);
* ```
*/
export function add(a: number, b: number) {
return a + b;
}
````
This is virtually transformed into:
```ts
import { assertEquals } from "jsr:@std/assert/equals";
import { add } from "files:///path/to/mod.ts";
Deno.test("mod.ts$2-7.ts", async () => {
assertEquals(add(1, 2), 3);
});
```
Note that a new import statement is inserted here to make `add` function
available. In a nutshell, all items exported from `mod.ts` become available in
the generated pseudo file with this automatic import insertion.
The intention behind this design is that, from library user's standpoint, it
should be very obvious that this `add` function is what this example code is
attached to. Also, if there is an explicit import statement like
`import { add } from "./mod.ts"`, this import path `./mod.ts` is not helpful for
doc readers because they will need to import it in a different way.
The automatic import insertion has some edge cases, in particular where there is
a local variable in a snippet with the same name as one of the exported items.
This case is addressed by employing swc's scope analysis (see test cases for
more details).
## "type-checking only" mode stays around
This change will likely impact a lot of existing doc tests in the ecosystem
because some doc tests rely on the fact that they are not evaluated - some cause
side effects if executed, some throw errors at runtime although they do pass the
type check, etc. To help those tests gradually transition to the ones runnable
with the new `deno test --doc`, we will keep providing the ability to run
type-checking only via `deno check --doc`. Additionally there is a `--doc-only`
option added to the `check` subcommand too, which is useful when you want to
type-check on code snippets in markdown files, as normal `deno check` command
doesn't accept markdown.
## Demo
https://github.com/user-attachments/assets/47e9af73-d16e-472d-b09e-1853b9e8f5ce
---
Closes #4716
The map field has been empty for years now and we don't want the emit
file to be exposed so it allows us to iterate on making the cache
faster. Additionally, it's racy/unreliable to rely on this information.
Instead, people should emit the TS files themselves using tools like
deno_emit, typescript, esbuild, etc.
Closes https://github.com/denoland/deno/issues/17703
Fixes https://github.com/denoland/deno/issues/25333.
We fall back to junctions if the error kind is `PermissionDenied` but
the std library actually sets the kind to `Uncategorized` if the symlink
fails due to insufficient privileges. This was causing the fallback to
not actually fall back in this case.
This commit enables the `log` feature for the `tracing` crate.
This allows us to examine additional detailed logs emitted by third party crates
that use `tracing` crate for logging by setting `RUST_LOG` env var or passing
`-L` option in command line.
Closes #25045
```
> deno upgrade
error: Unsupported lockfile version 'invalid'. Try upgrading Deno or recreating the lockfile.
V:\scratch
> V:\deno\target\debug\deno upgrade
Looking up latest version
Local deno version 1.45.3 is the most recent release
```
Closes #24517
Closes #20729
Adds much better support for the unstable Deno workspaces as well as
support for npm workspaces. npm workspaces is still lacking in that we
only install packages into the root node_modules folder. We'll make it
smarter over time in order for it to figure out when to add node_modules
folders within packages.
This includes a breaking change in config file resolution where we stop
searching for config files on the first found package.json unless it's
in a workspace. For the previous behaviour, the root deno.json needs to
be updated to be a workspace by adding `"workspace":
["./path-to-pkg-json-folder-goes-here"]`. See details in
https://github.com/denoland/deno_config/pull/66
Closes #24340
Closes #24159
Closes #24161
Closes #22020
Closes #18546
Closes #16106
Closes #24160
Previously various reads of files in `node_modules` would error on
invalid UTF-8. These were cases involving:
- reading package.json from Rust
- reading package.json from JS
- reading CommonJS files from JS
- reading CommonJS files from Rust (for ESM translation)
- reading ESM files from Rust
Introduces a `SyncReadAsyncWriteLock` to make it harder to write to the
npm resolution without first waiting async in a queue. For the npm
resolution, reading synchronously is fine, but when updating, someone
should wait async, clone the data, then write the data at the end back.
Hard linking (`linkat`) is ridiculously slow on mac. `copyfile` is
better, but what's even faster is `clonefile`. It doesn't have the space
savings that comes with hardlinking, but the performance difference is
worth it imo.
```
❯ hyperfine -i -p 'rm -rf node_modules/' '../../d7/target/release/deno cache npm:@11ty/eleventy' 'deno cache npm:@11ty/eleventy'
Benchmark 1: ../../d7/target/release/deno cache npm:@11ty/eleventy
Time (mean ± σ): 115.4 ms ± 1.2 ms [User: 27.2 ms, System: 87.3 ms]
Range (min … max): 113.7 ms … 117.5 ms 10 runs
Benchmark 2: deno cache npm:@11ty/eleventy
Time (mean ± σ): 619.3 ms ± 6.4 ms [User: 34.3 ms, System: 575.6 ms]
Range (min … max): 612.2 ms … 633.3 ms 10 runs
Summary
../../d7/target/release/deno cache npm:@11ty/eleventy ran
5.37 ± 0.08 times faster than deno cache npm:@11ty/eleventy
```
