This PR fixes #21658.
- `check` subcommand sees `exclude` option in `deno.json`. When some
paths passed with `check` command listed in `exclude`, they are ignored.
- When some files are listed in `exclude` and imported indirectly among
module graph, they are checked.
Harden the code that does permission checks to protect against
re-opening of stdin.
Code that runs FFI is vulnerable to an attack where fd 0 is closed
during a permission check and re-opened with a file that contains a
positive response (ie: `y` or `A`). While FFI code is dangerous in
general, we can make it more difficult for FFI-enabled code to bypass
additional permission checks.
- Checks to see if the underlying file for stdin has changed from the
start to the end of the permission check (detects races)
- Checks to see if the message is excessively long (lowering the window
for races)
- Checks to see if stdin and stderr are still terminals at the end of
the function (making races more difficult)
Fixes #20516
Follow up to #21747 and #21746
This tackles the last point of #20516 where certain inputs weren't
accepted in the other zlib methods
This adds the `toU8` conversion of `_brotli` to `_zlib.mjs`, when we
create the ZLibBuffer, we'll sanitize the input. I noticed that the
async had no handler for `string` input so I added that as well.
Fixes https://github.com/denoland/deno/issues/21734
Changes:
- Use default encode when options do not provide a encode callback.
- Remove internal TS for `node:querystring`. Its not helping catching
bugs like this because of invalid type assumptions and use of `any`,
more of a maintenance burden.
This fixes point 3 of https://github.com/denoland/deno/issues/20516
This PR creates consistency between the sync and async versions of the
brotli compress where we will always return a buffer like Node.
This fixes point 2 of #20516
This adds a conversion from Dataview/Buffer by returning `obj.buffer`
which can be converted to a `UInt8Array`.
Question: Regarding point 4 of the mentioned issue would it be
appropriate to copy the toU8 helper to the `zlib.mjs` methods?
These tests started failing on CI on Dec 28th, 2023 returning
ENOTFOUND. It's unclear what's going on, since `dig -x
8.8.8.8.in-addr.arpa`.
It needs a deeper investigation, but I want to unblock main branch
in the meantime.
Main change is that:
- "hyper" has been renamed to "hyper_v014" to signal that it's legacy
- "hyper1" has been renamed to "hyper" and should be the default
`opAsync` requires a lookup by name on each async call. This is a
mechanical translation of all opAsync calls to ensureFastOps.
The `opAsync` API on Deno.core will be removed at a later time.
When we migrate to op-import-per-extension, we will want to ensure that
ops have one and only one place where they are imported. This tackles
the ops that are imported via `ensureFastOps`, but does not yet tackle
direct `ops` imports.
Landing ahead of https://github.com/denoland/deno_core/pull/393
Node HTTP/2 was using the default h2 `Bytes` datatype when we can be
making using of `BufView` like we do in `Deno.serve`.
`fetch` and `Deno.serverHttp` can't make use of `BufView` because they
are using `reqwest` which is stuck on hyper 0.x at this time.
This commit adds a way to connect to the TS compiler host that is run
as part of the "deno lsp" subcommand. This can be done by specifying
"DENO_LSP_INSPECTOR" variable.
---------
Co-authored-by: Nayeem Rahman <nayeemrmn99@gmail.com>
This PR implements the child_process IPC pipe between parent and child.
The implementation uses Windows named pipes created by parent and passes
the inheritable file handle to the child.
I've also replace parts of the initial implementation which passed the
raw parent fd to JS with resource ids instead. This way no file handle
is exposed to the JS land (both parent and child).
`IpcJsonStreamResource` can stream upto 800MB/s of JSON data on Win 11
AMD Ryzen 7 16GB (without `memchr` vectorization)
`napi_open_handle_scope` was returning a bogus handle_scope and we were
trying to close it in `napi_close_handle_scope`.
This is a bit of a challenge to test, but the following testcase comes
from #21601 and appears to be fixed by this.
```
import { decode } from "https://deno.land/std@0.209.0/encoding/base64.ts";
import sharp from "npm:sharp";
const img = 'iVBORw0KGgoAAAANSUhEUgAAAQAAAAEAAQMAAABmvDolAAAAA1BMVEWq09/P7Lz1AAAAH0lEQVRoge3BAQ0AAADCoPdPbQ43oAAAAAAAAAAAvg0hAAABmmDh1QAAAABJRU5ErkJggg==';
Deno.test("async", async () => {
const id = setTimeout(() => Deno.exit(1), 1000);
await sharp(decode(img)).toBuffer();
await sharp(decode(img)).toBuffer();
clearTimeout(id);
});
```
Fixes https://github.com/denoland/deno/issues/21594
I verified locally that this fixes the problem. I'm working on testing
harness for Jupyter kernel to catch regressions like this and will
add it in a follow up PR.
Bumped versions for 1.39.0
Please ensure:
- [x] Target branch is correct (`vX.XX` if a patch release, `main` if
minor)
- [x] Crate versions are bumped correctly
- [x] deno_std version is incremented in the code (see
`cli/deno_std.rs`)
- [x] Releases.md is updated correctly (think relevancy and remove
reverts)
To make edits to this PR:
```shell
git fetch upstream release_1_39.0 && git checkout -b release_1_39.0 upstream/release_1_39.0
```
cc @mmastrac
---------
Co-authored-by: mmastrac <mmastrac@users.noreply.github.com>
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
This change deprecates `Deno.Reader`, `Deno.ReaderSync`, `Deno.Writer`,
`Deno.WriterSync` and `Deno.Closer` in favour of the [Web Streams
API](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API).
After discussing with Yoshiya, we both thought now might be the right
time to deprecate these interfaces with v2 getting closer.
This PR implements the Node child_process IPC functionality in Deno on
Unix systems.
For `fd > 2` a duplex unix pipe is set up between the parent and child
processes. Currently implements data passing via the channel in the JSON
serialization format.
Part 2 of removing middleware.
This is somewhat awkward because `V8CrossThreadTaskSpawner` requires
tasks to be `Send`, but NAPI makes heavy use of `!Send` pointers. In
addition, Rust causes a closure to be `!Send` if you pull a `!Send`
value out of a struct.
---------
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Adds performance measurements for all ops used by the LSP. Also changes
output of "Language server status" page to include more precise
information.
Current suspicion is that computing "script version" takes a long time
for some users.
This problem occurred trying to load tensorflow.js
```
> import * as tf from 'npm:@tensorflow/tfjs';
Uncaught SyntaxError: Identifier 'mod' has already been declared at file:///Users/ry/Library/Caches/deno/npm/registry.npmjs.org/@tensorflow/tfjs/4.14.0/dist/tf.node.js:167:14
at async <anonymous>:1:33
```
This commit adds a no-op flushHeaders method to the ServerResponse
object. It is a nop because the ServerResponse implementation is based
on top of the Deno server API instead of the Node `OutgoingMessage`
base.
Fixes #21509
This fixes #21434 for `BroadcastChannel` and `WebSocketStream`.
`--unstable` still enable both, but granular unstable flags now also
work:
* `--unstable-net` now enables `WebSocketStream`.
* `--unstable-broadcast-channel` now enables `BroadcastChannel`.
* Additionally, there are now tests for all granular unstable flags.
Since `unsafe-proto` already had tests, so I didn't add any for this
one.
It also introduces a map to keep track of granular unstable ids without
having to sync multiple places.
This commit brings back usage of primordials in "40_testing.js" by
turning it back into an ES module and using new "lazy loading" functionality
of ES modules coming from "deno_core".
The same approach was applied to "40_jupyter.js".
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Adds an `--unstable-sloppy-imports` flag which supports the
following for `file:` specifiers:
* Allows writing `./mod` in a specifier to do extension probing.
- ex. `import { Example } from "./example"` instead of `import { Example
} from "./example.ts"`
* Allows writing `./routes` to do directory extension probing for files
like `./routes/index.ts`
* Allows writing `./mod.js` for *mod.ts* files.
This functionality is **NOT RECOMMENDED** for general use with Deno:
1. It's not as optimal for perf:
https://marvinh.dev/blog/speeding-up-javascript-ecosystem-part-2/
1. It makes tooling in the ecosystem more complex in order to have to
understand this.
1. The "Deno way" is to be explicit about what you're doing. It's better
in the long run.
1. It doesn't work if published to the Deno registry because doing stuff
like extension probing with remote specifiers would be incredibly slow.
This is instead only recommended to help with migrating existing
projects to Deno. For example, it's very useful for getting CJS projects
written with import/export declaration working in Deno without modifying
module specifiers and for supporting TS ESM projects written with
`./mod.js` specifiers.
This feature will output warnings to guide the user towards correcting
their specifiers. Additionally, quick fixes are provided in the LSP to
update these specifiers:
Landing changes required for
https://github.com/denoland/deno_core/pull/359
We needed to update 99_main.js and a whole load of tests.
API changes:
- setPromiseRejectCallback becomes setUnhandledPromiseRejectionHandler.
The function is now called from eventLoopTick.
- The promiseRejectMacrotaskCallback no longer exists, as this is
automatically handled in eventLoopTick.
- ops.op_dispatch_exception now takes a second parameter: in_promise.
The preferred way to call this op is now reportUnhandledException or
reportUnhandledPromiseRejection.
This commit stabilizes "Deno.HttpServer.shutdown" API as well as
Unix socket support in "Deno.serve" API.
---------
Co-authored-by: Yoshiya Hinosawa <stibium121@gmail.com>
A bit hacky, but it works. Essentially, this will check for all the
scripts in the node_modules/.bin directory then force them to run with
Deno via deno_task_shell.
This commit adds a method of `Symbol.dispose` to the object returned
from `Deno.createHttpClient`, so we can make use of [explicit resource
management](https://github.com/tc39/proposal-explicit-resource-management)
by declaring it with `using`.
This commit adds support for a new `kv.watch()` method that allows
watching for changes to a key-value pair. This is useful for cases
where you want to be notified when a key-value pair changes, but
don't want to have to poll for changes.
---------
Co-authored-by: losfair <zhy20000919@hotmail.com>
Add support for signing with a RSA PEM private key: `pkcs8` and `pkcs1`.
Fixes https://github.com/denoland/deno/issues/18972
Ref #21124
Verified fix with `npm:sshpk`. Unverfied but fixes
`npm:google-auth-library`, `npm:web-push` & `oracle/oci-typescript-sdk`
---------
Signed-off-by: Divy Srivastava <dj.srivastava23@gmail.com>
This commit removes some of the technical debt related
to snapshotting JS code:
- "cli/ops/mod.rs" and "cli/build.rs" no longer define "cli" extension
which was not required anymore
- Cargo features for "deno_runtime" crate have been unified in
"cli/Cargo.toml"
- "cli/build.rs" uses "deno_runtime::snapshot::create_runtime_snapshot"
API
instead of copy-pasting the code
- "cli/js/99_main.js" was completely removed as it's not necessary
anymore
Towards https://github.com/denoland/deno/issues/21137
This PR causes Deno to include more files in the graph based on how a
template literal looks that's provided to a dynamic import:
```ts
const file = await import(`./dir/${expr}`);
```
In this case, it will search the `dir` directory and descendant
directories for any .js/jsx/etc modules and include them in the graph.
To opt out of this behaviour, move the template literal to a separate
line:
```ts
const specifier = `./dir/${expr}`
const file = await import(specifier);
```
Switch `ext/fetch` over to `resourceForReadableStream` to simplify and
unify implementation with `ext/serve`. This allows us to work in Rust
with resources only.
Two additional changes made to `resourceForReadableStream` were
required:
- Add an optional length to `resourceForReadableStream` which translates
to `size_hint`
- Fix a bug where writing to a closed stream that was full would panic
This commit changes LSP log names by prefixing them, we now have these
prefixes:
- `lsp.*` - requests coming from the client
- `tsc.request.*` - requests coming from clients that are routed to TSC
- `tsc.op.*` - ops called by the TS host
- `tsc.host.*` - requests that call JavaScript runtime that runs
TypeScript compiler host
Additionall `Performance::mark` was split into `Performance::mark` and
`Performance::mark_with_args` to reduce verbosity of code and logs.
