This completely rewrites how we handle key material in ext/node. Changes
in this
PR:
- **Signing**
- RSA
- RSA-PSS 🆕
- DSA 🆕
- EC
- ED25519 🆕
- **Verifying**
- RSA
- RSA-PSS 🆕
- DSA 🆕
- EC 🆕
- ED25519 🆕
- **Private key import**
- Passphrase encrypted private keys 🆕
- RSA
- PEM
- DER (PKCS#1) 🆕
- DER (PKCS#8) 🆕
- RSA-PSS
- PEM
- DER (PKCS#1) 🆕
- DER (PKCS#8) 🆕
- DSA 🆕
- EC
- PEM
- DER (SEC1) 🆕
- DER (PKCS#8) 🆕
- X25519 🆕
- ED25519 🆕
- DH
- **Public key import**
- RSA
- PEM
- DER (PKCS#1) 🆕
- DER (PKCS#8) 🆕
- RSA-PSS 🆕
- DSA 🆕
- EC 🆕
- X25519 🆕
- ED25519 🆕
- DH 🆕
- **Private key export**
- RSA 🆕
- DSA 🆕
- EC 🆕
- X25519 🆕
- ED25519 🆕
- DH 🆕
- **Public key export**
- RSA
- DSA 🆕
- EC 🆕
- X25519 🆕
- ED25519 🆕
- DH 🆕
- **Key pair generation**
- Overhauled, but supported APIs unchanged
This PR adds a lot of new individual functionality. But most importantly
because
of the new key material representation, it is now trivial to add new
algorithms
(as shown by this PR).
Now, when adding a new algorithm, it is also widely supported - for
example
previously we supported ED25519 key pair generation, but we could not
import,
export, sign or verify with ED25519. We can now do all of those things.
Uses [sui](https://github.com/littledivy/sui) to inject metadata as a
custom section in the denort binary.
Metadata is stored as a Mach-O segment on macOS and PE `RT_RCDATA`
resource on Windows.
Fixes #11154
Fixes https://github.com/denoland/deno/issues/17753
```cpp
deno compile app.tsx
# on macOS
codesign --sign - ./app
# on Windows
signtool sign /fd SHA256 .\app.exe
```
---------
Signed-off-by: Divy Srivastava <dj.srivastava23@gmail.com>
- upgrade to v8 12.8
- optimizes DataView bigint methods
- fixes global interceptors
- includes CPED methods for ALS
- fix global resolution
- makes global resolution consistent using host_defined_options.
originally a separate patch but due to the global interceptor bug it
needs to be included in this pr for all tests to pass.
Fixes https://github.com/denoland/deno/issues/24756. Fixes
https://github.com/denoland/deno/issues/24796.
This also gets vitest working when using
[`--pool=forks`](https://vitest.dev/guide/improving-performance#pool)
(which is the default as of vitest 2.0). Ref
https://github.com/denoland/deno/issues/23882.
---
This PR resolves a handful of issues with child_process IPC. In
particular:
- We didn't support sending typed array views over IPC
- Opening an IPC channel resulted in the event loop never exiting
- Sending a `null` over IPC would terminate the channel
- There was some UB in the read implementation (transmuting an `&[u8]`
to `&mut [u8]`)
- The `send` method wasn't returning anything, so there was no way to
signal backpressure (this also resulted in the benchmark
`child_process_ipc.mjs` being misleading, as it tried to respect
backpressure. That gave node much worse results at larger message sizes,
and gave us much worse results at smaller message sizes).
- We weren't setting up the `channel` property on the `process` global
(or on the `ChildProcess` object), and also didn't have a way to
ref/unref the channel
- Calling `kill` multiple times (or disconnecting the channel, then
calling kill) would throw an error
- Node couldn't spawn a deno subprocess and communicate with it over IPC
This commit duplicates ops from "ext/fetch" to "ext/node" to
kick off a bigger rewrite of "node:http".
Most of duplication is temporary and will be removed as these
ops evolve.
Sending ALPN to a proxy, and then when tunneling, requires better
juggling of TLS configs. This improves the choice of TLS config in the
proxy connector, based on what reqwest does. It also includes some
`ext/fetch/tests.rs` that check the different combinations.
Fixes #24632
Fixes #24691
This is in preparation for extracting out node resolution code from
ext/node (which is something I'm going to do gradually over time).
Uses https://github.com/denoland/deno_package_json
This is the release commit being forwarded back to main for 1.45.3
---------
Co-authored-by: bartlomieju <bartlomieju@users.noreply.github.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This PR will bump the deno_cache_dir dependency from 0.10.0 to 0.10.2.
This will make the cache manifest at manifest.json more deterministic
(see denoland/deno_cache_dir#53 )
<!--
Before submitting a PR, please read
https://docs.deno.com/runtime/manual/references/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Originally landed in
f6fd6619e7.
Reverted in https://github.com/denoland/deno/pull/24574.
This reland contains a fix that sends "Accept: */*" header for calls made
from "FileFetcher". Absence of this header made downloading source code
from JSR broken. This is tested by ensuring this header is present in the
test server that servers JSR packages.
---------
Co-authored-by: Sean McArthur <sean@seanmonstar.com>
This makes bare specifiers for npm packages work when inside a
workspace, which emulates the same behaviour as when there's a
node_modules directory. The bare specifier can be overwritten by
specifying an import map entry or package.json dependency entry.
* https://github.com/denoland/deno_config/pull/88
Closes #24605
This commit re-implements `ext/fetch` and all dependent crates
using `hyper` and `hyper-util`, instead of `reqwest`.
The reasoning is that we want to have greater control and access
to low level `hyper` APIs when implementing `fetch` API as well
as `node:http` module.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This is a temporary fix, which is not perfect - specifying
`--import-map`
will break resolution of packages defined in `workspace` setting, but
erroring on `--import-map` currently causes regression in code that
worked fine in v1.44.x.
This commit fixes memory leak described in
https://github.com/denoland/deno/issues/24380.
This is done by upgrading following crates:
- deno_ast
- deno_graph
- eszip
- dprint-plugin-typescript
- deno_lint
- deno_doc
- deno_emit
Adds support for running npm package lifecycle scripts, opted into via a
new `--allow-scripts` flag.
With this PR, when running `deno cache` (or `DENO_FUTURE=1 deno
install`) you can specify the `--allow-scripts=pkg1,pkg2` flag to run
lifecycle scripts attached to the given packages.
Note at the moment this only works when `nodeModulesDir` is true (using
the local resolver).
When a package with un-run lifecycle scripts is encountered, we emit a
warning suggesting things may not work and to try running lifecycle
scripts. Additionally, if a package script implicitly requires
`node-gyp` and it's not found on the system, we emit a warning.
Extra things in this PR:
- Extracted out bits of `task.rs` into a separate module for reuse
- Added a couple fields to `process.config` in order to support
`node-gyp` (it relies on a few variables being there)
- Drive by fix to downloading new npm packages to test registry
---
TODO:
- [x] validation for allow-scripts args (make sure it looks like an npm
package)
- [x] make allow-scripts matching smarter
- [ ] figure out what issues this closes
---
Review notes:
- This adds a bunch of deps to our test registry due to using
`node-gyp`, so it's pretty noisy
This adds object config for the workspace config:
```json
{
"workspace": {
"members": ["./member-1", "./member-2"]
}
}
```
This is a more verbose version of `"workspace": ["./member-1",
"./member-2"]`. Although we don't need it at the moment, it makes the
naming of `"workspace"` more clear and leaves the object open for more
config in the future.
Closes https://github.com/denoland/deno/issues/24456
Adds much better support for the unstable Deno workspaces as well as
support for npm workspaces. npm workspaces is still lacking in that we
only install packages into the root node_modules folder. We'll make it
smarter over time in order for it to figure out when to add node_modules
folders within packages.
This includes a breaking change in config file resolution where we stop
searching for config files on the first found package.json unless it's
in a workspace. For the previous behaviour, the root deno.json needs to
be updated to be a workspace by adding `"workspace":
["./path-to-pkg-json-folder-goes-here"]`. See details in
https://github.com/denoland/deno_config/pull/66
Closes #24340
Closes #24159
Closes #24161
Closes #22020
Closes #18546
Closes #16106
Closes #24160
Changes in this PR:
- Added new fixed size hash algorithms (blake2b512, blake2s256,
sha512-224, sha512-256, sha3-224, sha3-256, sha3-384, sha3-512, sm3)
- Added variable size hash algorithms (the concept), with the algorithms
shake128 and shake256
- Use cppgc instead of resources for the hasher
- Enable Node's crypto.Hash tests and fix found bugs
This is the release commit being forwarded back to main for 1.44.3
Co-authored-by: bartlomieju <bartlomieju@users.noreply.github.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
<!--
Before submitting a PR, please read
https://docs.deno.com/runtime/manual/references/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Fixes #20663.
---------
Co-authored-by: Asher Gomez <ashersaupingomez@gmail.com>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
This commit updates Deno to use `reqwest` at 0.12.4
and `rustls` at 0.22. Other related crates were updated
as well to match versions accepted by `reqwest` and `rustls`.
Note: we are not using the latest available `rustls` yet,
but this upgrade was non-trivial already, so a bump to
0.23 for `rustls` will be done in a separate commit.
Closes #23370
---------
Signed-off-by: Ryan Dahl <ry@tinyclouds.org>
Signed-off-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Also removes permissions being passed in for node resolution. It was
completely useless because we only checked it for reading package.json
files, but Deno reading package.json files for resolution is perfectly
fine.
My guess is this is also a perf improvement because Deno is doing less
work.
This is the release commit being forwarded back to main for 1.44.1
Co-authored-by: devsnek <devsnek@users.noreply.github.com>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
This makes `DOMException`'s `stack` property behave the same as native
errors' – `stack` is now an own accessor property on every instance, and
the getter calls `Error.prepareStackTrace`.
Upgrades `deno_core` to 0.284.0.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Brings in:
* More fully typed structures (for when we get to implementing more)
* `with_metadata`, `with_buffers`, etc. from
https://github.com/runtimed/runtimed/pull/99
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
In https://github.com/denoland/deno/pull/23955 we changed the sqlite db
journal mode to WAL. This causes issues when someone is running an old
version of Deno using TRUNCATE and a new version because the two fight
against each other.
This patch fixes stdin read hanging on user input when switching tty
mode on Windows
Fixes #21111
On Windows, when switching from line to raw mode:
- Cancel ongoing console read by writing a return keypress to its input
buffer. This blocks the main thread until any ongoing read has been
cancelled to prevent interference with the screen state.
- On the read thread, restore the cursor position to where it was before
writing the enter, undoing its effect on the screen state.
- Restart reading and notify the main thread.
This brings in [`runtimelib`](https://github.com/runtimed/runtimed) to
use:
## Fully typed structs for Jupyter Messages
```rust
let msg = connection.read().await?;
self
.send_iopub(
runtimelib::Status::busy().as_child_of(msg),
)
.await?;
```
## Jupyter paths
Jupyter paths are implemented in Rust, allowing the Deno kernel to be
installed completely via Deno without a requirement on Python or
Jupyter. Deno users will be able to install and use the kernel with just
VS Code or other editors that support Jupyter.
```rust
pub fn status() -> Result<(), AnyError> {
let user_data_dir = user_data_dir()?;
let kernel_spec_dir_path = user_data_dir.join("kernels").join("deno");
let kernel_spec_path = kernel_spec_dir_path.join("kernel.json");
if kernel_spec_path.exists() {
log::info!("✅ Deno kernel already installed");
Ok(())
} else {
log::warn!("ℹ️ Deno kernel is not yet installed, run `deno jupyter --install` to set it up");
Ok(())
}
}
```
Closes https://github.com/denoland/deno/issues/21619
Related: https://github.com/denoland/eszip/pull/181
eszip < v0.69.0 hashes all its contents to ensure data integrity. This
feature is not necessary in Deno CLI as the binary integrity guarantee
is deemed an external responsibility (ie it is to be assumed that, if
necessary, the compiled binary will be checksumed externally prior to
being executed).
eszip >= v0.69.0 no longer performs this checksum by default. This
reduces the cold-start time of the compiled binaries, proportionally to
their size.
Precursor to #23236
This implements the SNI features, but uses private symbols to avoid
exposing the functionality at this time. Note that to properly test this
feature, we need to add a way for `connectTls` to specify a hostname.
This is something that should be pushed into that API at a later time as
well.
```ts
Deno.test(
{ permissions: { net: true, read: true } },
async function listenResolver() {
let sniRequests = [];
const listener = Deno.listenTls({
hostname: "localhost",
port: 0,
[resolverSymbol]: (sni: string) => {
sniRequests.push(sni);
return {
cert,
key,
};
},
});
{
const conn = await Deno.connectTls({
hostname: "localhost",
[serverNameSymbol]: "server-1",
port: listener.addr.port,
});
const [_handshake, serverConn] = await Promise.all([
conn.handshake(),
listener.accept(),
]);
conn.close();
serverConn.close();
}
{
const conn = await Deno.connectTls({
hostname: "localhost",
[serverNameSymbol]: "server-2",
port: listener.addr.port,
});
const [_handshake, serverConn] = await Promise.all([
conn.handshake(),
listener.accept(),
]);
conn.close();
serverConn.close();
}
assertEquals(sniRequests, ["server-1", "server-2"]);
listener.close();
},
);
```
---------
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
1. Generally we should prefer to use the `log` crate.
2. I very often accidentally commit `eprintln`s.
When we should use `println` or `eprintln`, it's not too bad to be a bit
more verbose and ignore the lint rule.
**THIS PR HAS GIT CONFLICTS THAT MUST BE RESOLVED**
This is the release commit being forwarded back to main for 1.43.2
Please ensure:
- [x] Everything looks ok in the PR
- [x] The release has been published
To make edits to this PR:
```shell
git fetch upstream forward_v1.43.2 && git checkout -b forward_v1.43.2 upstream/forward_v1.43.2
```
Don't need this PR? Close it.
cc @nathanwhit
Co-authored-by: nathanwhit <nathanwhit@users.noreply.github.com>
Co-authored-by: Nathan Whitaker <nathan@deno.com>
This PR implements the changes we plan to make to `deno install` in deno
2.0.
- `deno install` without arguments caches dependencies from
`package.json` / `deno.json` and sets up the `node_modules` folder
- `deno install <pkg>` adds the package to the config file (either
`package.json` or `deno.json`), i.e. it aliases `deno add`
- `deno add` can also add deps to `package.json` (this is gated behind
`DENO_FUTURE` due to uncertainty around handling projects with both
`deno.json` and `package.json`)
- `deno install -g <bin>` installs a package as a globally available
binary (the same as `deno install <bin>` in 1.0)
---------
Co-authored-by: Nathan Whitaker <nathan@deno.com>
Allows writing named sub-tests. These are:
1. Filterable on the command line via `cargo test ...`
2. Run in parallel
3. Use a fresh temp and deno dir for each test (unlike steps)
