This commit re-implements `ext/fetch` and all dependent crates
using `hyper` and `hyper-util`, instead of `reqwest`.
The reasoning is that we want to have greater control and access
to low level `hyper` APIs when implementing `fetch` API as well
as `node:http` module.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
NPM inserts a default install script when a package has a `binding.gyp`
file.
It's possible, however, for the package to exclude the `binding.gyp`
file when they publish, and in this case the install script will never
succeed for a user of the package.
This happens with `fsevents`, for instance. They don't include the
`binding.gyp` file in their published tarball, but the default install
script appears in the manifest served by `npm`.
This causes us to warn that `fsevents` has an install script, but when
you try to run it it fails due to `binding.gyp` not existing.
This is a temporary fix, which is not perfect - specifying
`--import-map`
will break resolution of packages defined in `workspace` setting, but
erroring on `--import-map` currently causes regression in code that
worked fine in v1.44.x.
Previously when we printed out the packages that skipped install
scripts, we didn't prefix them with `npm:`. When you pass
`--allow-scripts` though, we require `npm:`, which means you can't just
copy paste the package name from the warning message.
Adds support for running npm package lifecycle scripts, opted into via a
new `--allow-scripts` flag.
With this PR, when running `deno cache` (or `DENO_FUTURE=1 deno
install`) you can specify the `--allow-scripts=pkg1,pkg2` flag to run
lifecycle scripts attached to the given packages.
Note at the moment this only works when `nodeModulesDir` is true (using
the local resolver).
When a package with un-run lifecycle scripts is encountered, we emit a
warning suggesting things may not work and to try running lifecycle
scripts. Additionally, if a package script implicitly requires
`node-gyp` and it's not found on the system, we emit a warning.
Extra things in this PR:
- Extracted out bits of `task.rs` into a separate module for reuse
- Added a couple fields to `process.config` in order to support
`node-gyp` (it relies on a few variables being there)
- Drive by fix to downloading new npm packages to test registry
---
TODO:
- [x] validation for allow-scripts args (make sure it looks like an npm
package)
- [x] make allow-scripts matching smarter
- [ ] figure out what issues this closes
---
Review notes:
- This adds a bunch of deps to our test registry due to using
`node-gyp`, so it's pretty noisy
Adds much better support for the unstable Deno workspaces as well as
support for npm workspaces. npm workspaces is still lacking in that we
only install packages into the root node_modules folder. We'll make it
smarter over time in order for it to figure out when to add node_modules
folders within packages.
This includes a breaking change in config file resolution where we stop
searching for config files on the first found package.json unless it's
in a workspace. For the previous behaviour, the root deno.json needs to
be updated to be a workspace by adding `"workspace":
["./path-to-pkg-json-folder-goes-here"]`. See details in
https://github.com/denoland/deno_config/pull/66
Closes #24340
Closes #24159
Closes #24161
Closes #22020
Closes #18546
Closes #16106
Closes #24160
Prevent panic when enabling a feature that is already enabled by
removing duplicate features.
Closes #22015
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Closes #18296.
Adds a `--frozen` (alias `--frozen-lockfile`) flag that errors out if
the lockfile is out of date. This is useful for running in CI (where an
out of date lockfile is usually a mistake) or to prevent accidental
changes in dependencies.
This commit adds discovery of `.npmrc` files in user's homedir.
This is not a perfect fix as it doesn't merge multiple `.npmrc` files
together as per https://github.com/denoland/deno/issues/23954
but allows to fallback if no `.npmrc` file is discovered in the project
root.
This commit updates Deno to use `reqwest` at 0.12.4
and `rustls` at 0.22. Other related crates were updated
as well to match versions accepted by `reqwest` and `rustls`.
Note: we are not using the latest available `rustls` yet,
but this upgrade was non-trivial already, so a bump to
0.23 for `rustls` will be done in a separate commit.
Closes #23370
---------
Signed-off-by: Ryan Dahl <ry@tinyclouds.org>
Signed-off-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Co-authored-by: Ryan Dahl <ry@tinyclouds.org>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Fixes a regression introduced in
https://github.com/denoland/deno/pull/24170, where we wouldn't actually
set up the node modules dir on `deno install` if there was an up to date
deno lockfile present.
Previously we were relying on the fact that resolving pending module
resolution called `cache_packages` (which sets up the node modules dir).
When pending resolutions were removed, and the `resolve_pending`
function with it, we also removed the `cache_packages` call needed to
set up node modules.
Previously various reads of files in `node_modules` would error on
invalid UTF-8. These were cases involving:
- reading package.json from Rust
- reading package.json from JS
- reading CommonJS files from JS
- reading CommonJS files from Rust (for ESM translation)
- reading ESM files from Rust
Fixes #24012.
In the case of multiple packages providing a binary with a same name, we
were basically leaving the results undefined (since we set up things in
parallel, and whichever got set up first won). In addition, we were
warning about these cases, even though it's a situation that's expected
to occur.
Instead, in the case of a collision in the binary names, we prefer the
binary provided by the package with the least depth in the dependency
tree.
While I was at it, I also took moved more code to `bin_entries.rs` since
it was starting to get a bit cluttered.
This commits adds the ability to set a would-be exit code
for the Deno process without forcing an immediate exit,
through the new `Deno.exitCode` API.
- **Implements `Deno.exitCode` getter and setter**: Adds support for
setting
and retrieving a would-be exit code via `Deno.exitCode`.
This allows for asynchronous cleanup before process termination
without immediately exiting.
- **Ensures type safety**: The setter for `Deno.exitCode` validates that
the provided value is a number, throwing a TypeError if not, to ensure
that
only valid exit codes are set.
Closes to #23605
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Enhanced warning message for --env flag with run and eval subcommands.
The commit is specifically made to address issue #23674 by improving the
warning messages that appear when using the --env flag with run or eval
subcommands in the following scenarios:
1. Missing environment file.
2. Incorrect syntax in the environment file content.
**Changes made**
- Distinguishes between cases of missing environment file and wrong
syntax in the environment file content.
- Shows a concise warning message to convey the case/issue occurred.
**Code changes & enhancements**
- Implemented a match statement to handle different types of errors
received while getting and parsing the file content to display a concise
warning message, rather than simple error check and then displaying the
same warning message for whatever the type of error is.
- Updated the related existing tests to reflect the new warning
messages.
- Added two test cases to cover the wrong environment file content
syntax with both run and eval subcommands.
**Impact**
The use of --env flag with both run/eval would be more user-friendly as
it gives a precise description of what is not right when using
incorrectly.
If you could give it a look, @dsherret , I appreciate your feedback on
these changes.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit adds initial support for ".npmrc" files.
Currently we only discover ".npmrc" files next to "package.json" files
and discovering these files in user home dir is left for a follow up.
This pass supports "_authToken" and "_auth" configuration
for providing authentication.
LSP support has been left for a follow up PR.
Towards https://github.com/denoland/deno/issues/16105
Fixes #23571.
Previously, we required a `deno.json` to be present (or the `--lock`
flag) in order for us to resolve a `deno.lock` file. This meant that if
you were using deno in an npm-first project deno wouldn't use a
lockfile.
Additionally, while I was fixing that, I discovered there were a couple
bugs keeping the future `install` command from using a lockfile.
With this PR, `install` will actually resolve the lockfile (or create
one if not present), and update it if it's not up-to-date. This also
speeds up `deno install`, as we can use the lockfile to skip work during
npm resolution.
By default, uses a 60 second timeout, backing off 2x each time (can be
overridden using the hidden `DENO_SLOW_TEST_TIMEOUT` which we implement
only really for spec testing.
```
Deno.test(async function test() {
await new Promise(r => setTimeout(r, 130_000));
});
```
```
$ target/debug/deno test /tmp/test_slow.ts
Check file:///tmp/test_slow.ts
running 1 test from ../../../../../../tmp/test_slow.ts
test ...'test' is running very slowly (1m0s)
'test' is running very slowly (2m0s)
ok (2m10s)
ok | 1 passed | 0 failed (2m10s)
```
---------
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This brings in [`runtimelib`](https://github.com/runtimed/runtimed) to
use:
## Fully typed structs for Jupyter Messages
```rust
let msg = connection.read().await?;
self
.send_iopub(
runtimelib::Status::busy().as_child_of(msg),
)
.await?;
```
## Jupyter paths
Jupyter paths are implemented in Rust, allowing the Deno kernel to be
installed completely via Deno without a requirement on Python or
Jupyter. Deno users will be able to install and use the kernel with just
VS Code or other editors that support Jupyter.
```rust
pub fn status() -> Result<(), AnyError> {
let user_data_dir = user_data_dir()?;
let kernel_spec_dir_path = user_data_dir.join("kernels").join("deno");
let kernel_spec_path = kernel_spec_dir_path.join("kernel.json");
if kernel_spec_path.exists() {
log::info!("✅ Deno kernel already installed");
Ok(())
} else {
log::warn!("ℹ️ Deno kernel is not yet installed, run `deno jupyter --install` to set it up");
Ok(())
}
}
```
Closes https://github.com/denoland/deno/issues/21619
Construct a new module graph container for workers instead of sharing it
with the main worker.
Fixes #17248
Fixes #23461
---------
Co-authored-by: David Sherret <dsherret@gmail.com>
This PR implements the changes we plan to make to `deno install` in deno
2.0.
- `deno install` without arguments caches dependencies from
`package.json` / `deno.json` and sets up the `node_modules` folder
- `deno install <pkg>` adds the package to the config file (either
`package.json` or `deno.json`), i.e. it aliases `deno add`
- `deno add` can also add deps to `package.json` (this is gated behind
`DENO_FUTURE` due to uncertainty around handling projects with both
`deno.json` and `package.json`)
- `deno install -g <bin>` installs a package as a globally available
binary (the same as `deno install <bin>` in 1.0)
---------
Co-authored-by: Nathan Whitaker <nathan@deno.com>
Allows writing named sub-tests. These are:
1. Filterable on the command line via `cargo test ...`
2. Run in parallel
3. Use a fresh temp and deno dir for each test (unlike steps)
This commit updates our testing npm registry to handle
additional `@denotest2` scope in addition to `@denotest`
scope. I might have to update it further in the future to handle
additional scopes, but it's good enough for now.
Part of #22607 (probably closes it, but I haven't done thorough testing)
Makes it so that `require.resolve` with `paths` specified will fallback
to using the global cache when the paths can't be found when using a
global cache (not when using a node_modules folder)
