Handles ASCCI espace chars in test and bench name making
test and bench reporting more reliable. This one is also tested
in the fixture of "node:test" module.
This commit moves `snapshot_from_lockfile` function to [deno_npm
crate](https://github.com/denoland/deno_npm). This allows this function
to be called outside Deno CLI (in particular, Deno Deploy).
Renames the unstable `deno_modules` directory and corresponding settings
to `vendor` after feedback. Also causes the vendoring of the
`node_modules` directory which can be disabled via
`--node-modules-dir=false` or `"nodeModulesDir": false`.
This changes the design of the manifest.json file to have a separate
"folders" map for mapping hashed directories. This allows, for example,
to add files in a folder like `http_localhost_8000/#testing_5de71/` and
have them be resolved automatically as long as their remaining
components are identity-mappable to the file system (not hashed). It
also saves space in the manifest.json file by only including the hashed
directory instead of each descendant file.
```
// manifest.json
{
"folders": {
"https://localhost/NOT_MAPPABLE/": "localhost/#not_mappable_5cefgh"
},
"modules": {
"https://localhost/folder/file": {
"headers": {
"content-type": "application/javascript"
}
},
}
}
// folder structure
localhost
- folder
- #file_2defn (note: I've made up the hashes in these examples)
- #not_mappable_5cefgh
- mod.ts
- etc.ts
- more_files.ts
```
This commit adds new "--deny-*" permission flags. These are complimentary to
"--allow-*" flags.
These flags can be used to restrict access to certain resources, even if they
were granted using "--allow-*" flags or the "--allow-all" ("-A") flag.
Eg. specifying "--allow-read --deny-read" will result in a permission error,
while "--allow-read --deny-read=/etc" will allow read access to all FS but the
"/etc" directory.
Runtime permissions APIs ("Deno.permissions") were adjusted as well, mainly
by adding, a new "PermissionStatus.partial" field. This field denotes that
while permission might be granted to requested resource, it's only partial (ie.
a "--deny-*" flag was specified that excludes some of the requested resources).
Eg. specifying "--allow-read=foo/ --deny-read=foo/bar" and then querying for
permissions like "Deno.permissions.query({ name: "read", path: "foo/" })"
will return "PermissionStatus { state: "granted", onchange: null, partial: true }",
denoting that some of the subpaths don't have read access.
Closes #18804.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Nayeem Rahman <nayeemrmn99@gmail.com>
This commit adds a "dot" reporter to "deno test" subcommand,
that can be activated using "--dot" flag.
It provides a concise output using:
- "." for passing test
- "," for ignored test
- "!" for failing test
User output is silenced and not printed to the console.
In non-TTY environments each result is printed on a separate line.
This commit provides basic polyfill for "node:test" module. Currently
only top-level "test" function is polyfilled, all remaining functions from
that module throw not implemented errors.
Closes https://github.com/denoland/deno/issues/17251
Closes #19970
This commits adds logic to retry failed module downloads once.
Both request and server errors are handled and the retry is done after
50 ms wait time.
I'm not sure why, but sending SIGABRT to Deno on my machine as part of
this test causes it to lock up very badly, leaving it in an unkillable
`UE+` state.
This showed up after #19333, but was not caused by it.
Closes #17589.
```ts
Deno.bench("foo", async (t) => {
const resource = setup(); // not included in measurement
t.start();
measuredOperation(resource);
t.end();
resource.close(); // not included in measurement
});
```
This PR fixes #19818. The problem was that the new InnerRequest class does not initialize the fields urlList and urlListProcessed that are used during a request clone. The solution aims to be straightforward by simply initializing the missing properties during the clone process. I also implemented a "cache" to the url getter of the new InnerRequest, avoiding the cost of calling op_http_get_request_method_and_url.
We weren't auto-discovering the deno.json in two cases:
1. A project that didn't have a deno.json and just added one.
2. After a syntax error in the deno.json.
This now rediscovers it in both these cases.
Closes https://github.com/denoland/vscode_deno/issues/867
Code run within Deno-mode and Node-mode should have access to a
slightly different set of globals. Previously this was done through a
compile time code-transform for Node-mode, but this is not ideal and has
many edge cases, for example Node's globalThis having a different
identity than Deno's globalThis.
This commit makes the `globalThis` of the entire runtime a semi-proxy.
This proxy returns a different set of globals depending on the caller's
mode. This is not a full proxy, because it is shadowed by "real"
properties on globalThis. This is done to avoid the overhead of a full
proxy for all globalThis operations.
The globals between Deno-mode and Node-mode are now properly segregated.
This means that code running in Deno-mode will not have access to Node's
globals, and vice versa. Deleting a managed global in Deno-mode will
NOT delete the corresponding global in Node-mode, and vice versa.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Aapo Alasuutari <aapo.alasuutari@gmail.com>
This commit adds some regression tests for using `jsxImportSource` in
the config file in combination with an import map.
These underlying issues were fixed by #15561.
Closes #13389
Closes #14723
---------
Co-authored-by: Aapo Alasuutari <aapo.alasuutari@gmail.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit stabilizes "Deno.serve()", which becomes the
preferred way to create HTTP servers in Deno.
Documentation was adjusted for each overload of "Deno.serve()"
API and the API always binds to "127.0.0.1:8000" by default.
This PR changes Web IDL interfaces to be declared with `var` instead of
`class`, so that accessing them via `globalThis` does not raise type
errors.
Closes #13390.
This is a fix for issue #19644, concerning the `parseCssColor` function
in the file `ext/console/01_console.js`. Changes made on lines
2756-2758. To sum it up:
> The internal `parseCssColor` function currently parses 3/4-digit hex
colors incorrectly. For example, it parses the string `#FFFFFF` as
`[255, 255, 255]` (as expected), but returns `[240, 240, 240]` for
`#FFF`, when it should return the same triplet as the former.
While it's not going to cause a fatal runtime error, it did bug me
enough to fix it real quick.
Fixes #19568
Values are not coerced to the desired type during deserialisation. This
makes serde_v8 stricter.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This prevents documents specified in a deno.json's "exclude" from being
pre-loaded by the lsp.
For example, someone may have something like:
```jsonc
// deno.json
{
"exclude": [
"dist" // build directory
]
}
```
Reduce the GC pressure from the websocket event method by splitting it
into an event getter and a buffer getter.
Before:
165.9k msg/sec
After:
169.9k msg/sec
This commit adds basic support for "node:http2" module. Not
all APIs have been yet implemented, but this change already
allows to use this module for some basic functions.
The "grpc" package is still not working, but it's a good stepping
stone.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
## WHY
ref: https://github.com/denoland/deno/issues/19165
The FileHandle class has many missing methods compared to node.
Add these.
## WHAT
- Add close method
---------
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
Related issue: https://github.com/denoland/deno/issues/19358.
This is a regression that seems to have been introduced in
https://github.com/denoland/deno/pull/18905. It looks to have been a
performance optimization.
The issue is probably easiest described with some code:
```ts
const target = new EventTarget();
const event = new Event("foo");
target.addEventListener("foo", () => {
console.log('base');
target.addEventListener("foo", () => {
console.log('nested');
});
});
target.dispatchEvent(event);
```
Essentially, the second event listener is being attached while the `foo`
event is still being dispatched. It should then not fire that second
event listener, but Deno currently does.
<!--
Before submitting a PR, please read https://deno.com/manual/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Internally, `node-tap` spawns a child process with `stdio: [0, 1, 2]`.
Whilst we don't support passing fd numbers as an argument so far, it
turns out that `[0, 1, 2]` is equivalent to `"inherit"` which we already
support. See: https://nodejs.org/api/child_process.html#optionsstdio
Mapping it to `"inherit"` is fine for us and gets us one step closer in
getting `node-tap` working. I'm now at the stage where already the
coverage table is shown 🎉
<!--
Before submitting a PR, please read https://deno.com/manual/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
## WHY
ref: https://github.com/denoland/deno/issues/19165
Node's fs/promises includes a FileHandle class, but deno does not. The
open function in Node's fs/promises returns a FileHandle, which provides
an IO interface to the file. However, deno's open function returns a
resource id.
### deno
```js
> const fs = await import("node:fs/promises");
undefined
> const file3 = await fs.open("./README.md");
undefined
> file3
3
> file3.read
undefined
Node:
```
### Node
```js
> const fs = await import("fs/promises");
undefined
> const file3 = await fs.open("./tests/e2e_unit/testdata/file.txt");
undefined
> file3
FileHandle {
_events: [Object: null prototype] {},
_eventsCount: 0,
_maxListeners: undefined,
close: [Function: close],
[Symbol(kCapture)]: false,
[Symbol(kHandle)]: FileHandle {},
[Symbol(kFd)]: 24,
[Symbol(kRefs)]: 1,
[Symbol(kClosePromise)]: null
}
> file3.read
[Function: read]
```
To be compatible with Node, deno's open function should also return a
FileHandle.
## WHAT
I have implemented the first step in adding a FileHandle.
- Changed the return value of the open function to a FileHandle object
- Implemented the readFile method in FileHandle
- Add test code
## What to do next
This PR is the first step in adding a FileHandle, and there are things
that should be done next.
- Add functionality equivalent to Node's FileHandle to FileHandle
(currently there is only readFile)
---------
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
For the first implementation of node:http2, we'll use the internal
version of `Deno.serve` which allows us to listen on a raw TCP
connection rather than a listener.
This is mostly a refactoring, and hooking up of `op_http_serve_on` that
was never previously exposed (but designed for this purpose).
This PR adds the missing `process.reallyExit()` method to node's
`process` object.
Was [pinged on
twitter](https://twitter.com/biwanczuk/status/1663326659787862017)
regarding running the `fastify` test suite in node. They use `node-tap`
which has been around arguably the longest of the test frameworks and
relies on a couple of old APIs. They have `signal-exit` as a dependency
which in turn [makes use of
`process.reallyExit()`](8fa7fc9a9c/src/index.ts (L19)).
That function cannot be found anywhere in their documentation, but
exists at runtime. See
6a6b3c5402/lib/internal/bootstrap/node.js (L172)
This doesn't yet make `node-tap` work, but gets us one step closer.
Part of #17840
I haven't made any changes or additions to the tests themselves, just
moved the tests over and updated to match.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Rather than disallowing `ext:` resolution, clear the module map after
initializing extensions so extension modules are anonymized. This
operation is explicitly called in `deno_runtime`. Re-inject `node:`
specifiers into the module map after doing this.
Fixes #17717.
If a symlink within the `node_modules` directory lies outside that
directory, it will now warn and inline the file. For directories, it
will just warn for now.
Probably fixes #19251 (I'm still unable to reproduce).
`isFile`, `isDirectory`, `isSymlink` are defined in `Deno.FileInfo`, but
`isBlockDevice`, `isCharacterDevice`, `isFIFO`, `isSocket` are not
defined.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This is what pnpm does and we were missing it. It makes modules work
which have a dependency on something, but don't say they have that
dependency, but that dep is still in the tree somewhere.
This commit fixes emitting "unhandledrejection" event when there are
"node:" or "npm:" imports.
Before this commit the Node "unhandledRejection" event was emitted
using a regular listener for Web "unhandledrejection" event. This
listener was installed before any user listener had a chance to be
installed which effectively prevent emitting "unhandledrejection"
events to user code.
Closes https://github.com/denoland/deno/issues/16928
When someone explicitly opts into using the node_modules dir via
`--node-modules-dir` or setting `"nodeModulesDir": true` in the
deno.json file, we should eagerly ensure a top level package.json
install is done on startup. This was initially always done when we added
package.json support and a package.json was auto-discovered, but scaled
it back to be lazily done when a bare specifier matched an entry in the
package.json because of how disruptive it was for people using Deno
scripts in Node projects. That said, it does not make sense for someone
to opt-into having deno control and use their node_modules directory and
not want a package.json install to occur. If such a rare scenario
exists, the `DENO_NO_PACKAGE_JSON=1` environment variable can be set.
Ideally, we would only ever use a node_modules directory with this
explicit opt-in so everything is very clear, but we still have this
automatic scenario when there's a package.json in order to make more
node projects work out of the box.
Noticed that we're checking more module paths than necessary. In
particular the module path array contains a couple of entries with a
duplicated `node_modules/node_modules` suffix.
```js
[
// ... more entries before here, where some also contain duplicate suffixes
"/Users/marvinhagemeister/dev/preact-render-to-string/node_modules/.deno/node_modules",
"/Users/marvinhagemeister/dev/preact-render-to-string/node_modules/node_modules", // <-- duplicate suffix
"/Users/marvinhagemeister/dev/preact-render-to-string/node_modules",
"/Users/marvinhagemeister/dev/node_modules",
"/Users/marvinhagemeister/node_modules",
"/Users/node_modules",
"/node_modules",
"/node_modules" // <-- duplicate entry
]
```
This was caused by a misunderstanding in how Rust's
[`Path::ends_with()`](https://doc.rust-lang.org/std/path/struct.Path.html#method.ends_with)
works. It's designed to match on whole path segments and the suffix
`/node_modules` is not that, except for the root entry. This meant that
our check for if the path already ended with `node_module` always
returned `false`. Removing the leading slash fixes that.
While we're at it, we can remove the last condition where we explicitly
added the root `/node_modules` entry since the while loop prior to that
takes care of it already.
Follow up to https://github.com/denoland/deno/pull/19084.
This commit adds support for globs in the configuration file as well
as CLI arguments for files.
With this change users can now use glob syntax for "include" and
"exclude" fields, like so:
```json
{
"lint": {
"include": [
"directory/test*.ts",
"other_dir/"
],
"exclude": [
"other_dir/foo*.ts",
"nested/nested2/*"
]
},
"test": {
"include": [
"data/test*.ts",
"nested/",
"tests/test[1-9].ts"
],
"exclude": [
"nested/foo?.ts",
"nested/nested2/*"
]
}
}
```
Or in CLI args like so:
```
// notice quotes here; these values will be passed to Deno verbatim
// and deno will perform glob expansion
$ deno fmt --ignore="data/*.ts"
$ deno lint "data/**/*.ts"
```
Closes https://github.com/denoland/deno/issues/17971
Closes https://github.com/denoland/deno/issues/6365
This adds support for the lockfile and node_modules directory to the
lsp.
In the case of the node_modules directory, it is only enabled when
explicitly opted into via `"nodeModulesDir": true` in the configuration
file. This is to reduce the language server automatically modifying the
node_modules directory when the user doesn't want it to.
Closes #16510
Closes #16373
This runs our `js_unit_tests` and `node_unit_tests` in parallel, one
rust test per JS unit test file. Some of our JS tests don't like running
in parallel due to port requirements, so this also makes those use a
specific port-per-file. This does not attempt to make the node-compat
tests work.
We don't need to use the `deno` command here to test kill permissions
and it's awkward to get right without passing `-A`. `cat` works, but for
platforms other than windows. This test should have plenty of coverage
on other platforms.
This commit changes the implementation of `ext/web` timers, by using
"op_void_async_deferred" for timeouts of 0ms.
0ms timeout is meant to be run at the end of the event loop tick and
currently Tokio timers that we use to back timeouts have at least 1ms
resolution. That means that 0ms timeout actually take >1ms. This
commit changes that and runs 0ms timeout at the end of the event
loop tick.
One consequence is that "unrefing" a 0ms timer will actually keep
the event loop alive (which I believe actually makes sense, the test
we had only worked because the timeout took more than 1ms).
Ref https://github.com/denoland/deno/issues/19034
We never properly added support for this. This fixes vendoring when it
has npm or node specifiers. Vendoring occurs by adding a
`"nodeModulesDir": true` property to deno.json then it uses a local
node_modules directory. This can be opted out by setting
`"nodeModulesDir": false` or running with `--node-modules-dir=false`.
Closes #18090
Closes #17210
Closes #17619
Closes #16778
This commit changes the return type of an unstable `Deno.serve()` API
to instead return a `Deno.Server` object that has a `finished` field.
This change is done in preparation to be able to ref/unref the HTTP
server.
This is a bit bare bones but gets `npm-run-all` working. For full stdio
compatibility with node more work is needed which is probably better
done in follow up PRs.
Fixes #19159
Note: If the package information has already been cached, then this
requires running with `--reload` or for the registry information to be
fetched some other way (ex. the cache busting).
Closes #15544
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit reimplements most of "node:http" client APIs using
"ext/fetch".
There is some duplicated code and two removed Node compat tests that
will be fixed in follow up PRs.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
