Note: tests are not the only part of the codebase that uses `std`. Other
parts, like `tools/`, do too. So, it could be argued that this is a
little misleading. Either way, I'm doing this as discussed with
@mmastrac.
This introduces the `denort` binary - a slim version of deno without
tooling. The binary is used as the default for `deno compile`.
Improves `deno compile` final size by ~2.5x (141 MB -> 61 MB) on Linux
x86_64.
This PR separates integration tests from CLI tests into a new project
named `cli_tests`. This is a prerequisite for an integration test runner
that can work with either the CLI binary in the current project, or one
that is built ahead of time.
## Background
Rust does not have the concept of artifact dependencies yet
(https://github.com/rust-lang/cargo/issues/9096). Because of this, the
only way we can ensure a binary is built before running associated tests
is by hanging tests off the crate with the binary itself.
Unfortunately this means that to run those tests, you _must_ build the
binary and in the case of the deno executable that might be a 10 minute
wait in release mode.
## Implementation
To allow for tests to run with and without the requirement that the
binary is up-to-date, we split the integration tests into a project of
their own. As these tests would not require the binary to build itself
before being run as-is, we add a stub integration `[[test]]` target in
the `cli` project that invokes these tests using `cargo test`.
The stub test runner we add has `harness = false` so that we can get
access to a `main` function. This `main` function's sole job is to
`execvp` the command `cargo test -p deno_cli`, effectively "calling"
another cargo target.
This ensures that the deno executable is always correctly rebuilt before
running the stub test runner from `cli`, and gets us closer to be able
to run the entire integration test suite on arbitrary deno executables
(and therefore split the build into multiple phases).
The new `cli_tests` project lives within `cli` to avoid a large PR. In
later PRs, the test data will be split from the `cli` project. As there
are a few thousand files, it'll be better to do this as a completely
separate PR to avoid noise.
We don't need to wget `libdl` because we can just copy the one from
within the sysroot, saving two network accesses.
This allows amd64 to use the same build strategy as arm64.
Follow-up from #22298: Use a sysroot to build ARM64 so we work all the
way back to Xenial.
We generate a sysroot ahead-of-time in the
https://github.com/denoland/deno_sysroot_build project and use that to
bootstrap a sysroot here.
This implements officially blessed and tested deno binaries for ARM64.
Thanks to @LukeChannings for his tireless work in maintaining the
deno-arm64 [1] repo, without which this project would have been far more
complicated. For those of you requiring support for older GLIBC
versions, that repo may still be required for the near future.
Limitations:
- This initial build is built on Ubuntu 22 using the stock GLIBC, which
will limit the utility of these binaries in certain use-cases (eg: early
versions of Ubuntu). We will attempt to support earlier versions of
ARM64 GLIBC in a later revision.
- Like the stock Linux x64 build, this is not a static build and
requires GLIBC. Running on Alpine will require installation of GLIBC.
Fixes #1846, #4862
[1] https://github.com/LukeChannings/deno-arm64
We run these on the free machines now.
Also cleans up some of our os and arch conditional step handling by
introducing a new `matrix.os` and `matrix.arch`.
<!--
Before submitting a PR, please read https://deno.com/manual/contributing
1. Give the PR a descriptive title.
Examples of good title:
- fix(std/http): Fix race condition in server
- docs(console): Update docstrings
- feat(doc): Handle nested reexports
Examples of bad title:
- fix #7123
- update docs
- fix bugs
2. Ensure there is a related issue and it is referenced in the PR text.
3. Ensure there are tests that cover the changes.
4. Ensure `cargo test` passes.
5. Ensure `./tools/format.js` passes without changing files.
6. Ensure `./tools/lint.js` passes.
7. Open as a draft PR if your work is still in progress. The CI won't
run
all steps, but you can add '[ci]' to a commit message to force it to.
8. If you would like to run the benchmarks on the CI, add the 'ci-bench'
label.
-->
Fixes https://github.com/denoland/deno/issues/21880
Bumped versions for 1.39.0
Please ensure:
- [x] Target branch is correct (`vX.XX` if a patch release, `main` if
minor)
- [x] Crate versions are bumped correctly
- [x] deno_std version is incremented in the code (see
`cli/deno_std.rs`)
- [x] Releases.md is updated correctly (think relevancy and remove
reverts)
To make edits to this PR:
```shell
git fetch upstream release_1_39.0 && git checkout -b release_1_39.0 upstream/release_1_39.0
```
cc @mmastrac
---------
Co-authored-by: mmastrac <mmastrac@users.noreply.github.com>
Co-authored-by: Matt Mastracci <matthew@mastracci.com>
This is the release commit being forwarded back to main for 1.38.1
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Co-authored-by: littledivy <littledivy@users.noreply.github.com>
This patch adds a `remote` backend for `ext/kv`. This supports
connection to Deno Deploy and potentially other services compatible with
the KV Connect protocol.
Currently the resolution for extension sources is different depending on
whether `include_js_files_for_snapshotting`
is enabled. If sources are embedded it uses `include_str!()` which is
module-relative. If sources are read at runtime paths are joined to
`CARGO_MANIFEST_DIR` and are package-relative. This makes them both
package-relative.
Fixes `cargo run -p deno_runtime --example extension_with_esm --features
include_js_files_for_snapshotting`.
This is a new op system that will eventually replace `#[op]`.
Features
- More maintainable, generally less-coupled code
- More modern Rust proc-macro libraries
- Enforces correct `fast` labelling for fast ops, allowing for visual
scanning of fast ops
- Explicit marking of `#[string]`, `#[serde]` and `#[smi]` parameters.
This first version of op2 supports integer and Option<integer>
parameters only, and allows us to start working on converting ops and
adding features.
This commit adds ability to print metrics of the Tokio
runtime to the console by passing "DENO_TOKIO_METRICS=1"
env var.
Metrics will be printed every second, but this can be changed
by "DENO_TOKIO_METRICS_INTERVAL=500" env var.
This commit fixes problem with loading N-API modules that use
the "old" way of registration (using "napi_module_register" API).
The slot was not cleared after loading modules, causing subsequent
calls that use the new way of registration (using
"napi_register_module_v1" API) to try and load the previous module.
Ref https://github.com/denoland/deno/issues/16460
---------
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
**THIS PR HAS GIT CONFLICTS THAT MUST BE RESOLVED**
This is the release commit being forwarded back to main for 1.33.4
Please ensure:
- [x] Everything looks ok in the PR
- [ ] The release has been published
To make edits to this PR:
```shell
git fetch upstream forward_v1.33.4 && git checkout -b forward_v1.33.4 upstream/forward_v1.33.4
```
Don't need this PR? Close it.
cc @levex
Co-authored-by: levex <levex@users.noreply.github.com>
Co-authored-by: Levente Kurusa <lkurusa@kernelstuff.org>
Upgrade `thiserror` to `1.40.0`.
Remove version pinning so that consumers of deno crates can install
newer versions of `thiserrors` without waiting for us to upgrade our
Cargo.toml.
Upgrade `deno_lockfile` to `0.14.0` to bring in `thiserror` upgrade, see
https://github.com/denoland/deno_lockfile/pull/1.
Deno does not cover storage explosion attacks from evaluated runtime
code.
I've chosen the following parts for this clarification:
- _Evaluated_ code - storage explosion attacks caused by services in
Deno such as the HTTP server should still be covered.
- Isolated - If the storage explosion attack can happen at arbitrary
different files, it may leave a much more lasting impact on a targeted
host system than on simply the Deno cache.
Towards #18455
This commit implements the keypair generation for asymmetric keys for
the `generateKeyPair` API.
See how key material is managed in this implementation:
https://www.notion.so/denolandinc/node-crypto-design-99fc33f568d24e47a5e4b36002c5325d?pvs=4
Private and public key encoding depend on `KeyObject#export` which is
not implemented. I've also skipped ED448 and X448 since we need a crate
for that in WebCrypto too.
Updates CI to use ubuntu-22.04-xl runners. A change to sysroot setup
was necessary that links `libdl.so.2` and `libdl.a` that are no longer
present on ubuntu-22.04.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
- relands #17872
- updates the timeouts to be re-configurable just for CI
- fixes `./tools/wpt.ts update`
- adds option not "ignore" during, applied to wpt epoch runs only
Seems like our caching was totally broken. We need to save the cache
after building and not before.
```
Warning: Path Validation Error: Path(s) specified in the action for caching do(es) not exist, hence no cache is being saved.
Warning: Cache save failed.
```
Run benchmarks on a bare metal server. The bot is a webhook on Deno
deploy which can provision a spot instance on Equinix Metal. The machine
type is `m3.small.x86` running a Ubuntu 22.04.
Commands:
`+bench` - Provision and schedule benchmarks for this PR.
`+bench status <id>` - Get current status of the metal instance.
The bot source is here: https://github.com/denoland/bench_bot
Currently, we use `-rdynamic` for exporting Node API symbols to the
symbol table. `-rdynamic` will export *all* symbols, that means
previously unused functions will not be optimized away introducing a lot
of binary bloat.
This patch uses `-exported_symbol` and `--export-dynamic-symbol` link
flags (not as universal as `-rdynamic`) to only mark Node API symbols to
be put in the dynamic symbol table.
