Fixes #26085.
Adds a basic retry utility with some defaults, starts off with a 100ms
wait, then 250ms, then 500ms
I've applied the retry in the http client, reusing an existing function,
so this also applies to retrying downloads of deno binaries in `upgrade`
and `compile`. I can make a separate function that doesn't retry so this
doesn't affect `upgrade` and `compile`, but it seemed desirable to have
retries there too, so I left it in.
Fixes #25998. Fixes https://github.com/denoland/deno/issues/25928.
Originally I was just going to make this an error message instead of a
panic, but once I got to a minimal repro I felt that this really should
work.
The panic occurs when you have `nodeModulesDir: manual` (or a
package.json present), and you have an npm package with a tag in your
deno.json (see the spec test that illustrates this).
This code path only actually executes when trying to choose an
appropriate package version from `node_modules/.deno`, so we should be
able to fix it by storing some extra data at install time.
The fix proposed here is to repurpose the `.initialized` file that we
store in `node_modules` to store the tags associated with a package.
Basically, if you have a version requirement with a tag (e.g.
`npm:chalk@latest`), when we set up the node_modules folder for that
package, we store the tag (`latest`) in `.initialized`. Then, when doing
BYONM resolution, if we have a version requirement with a tag, we read
that file and check if the tag is present.
The downside is that we do more work when setting up `node_modules`. We
_could_ do this only when BYONM is enabled, but that would have the
downside of needing to re-run `deno install` when you switch from auto
-> manual, though maybe that's not a big deal.
Fixes #25861.
Previously we were attempting to match the version requirement against
the version already present in `node_modules` root, and if they didn't
match we would create a node_modules dir in the workspace member's
directory with the dependency.
Aside from the fact that this caused the panic, on second thought it
just doesn't make sense in general. We shouldn't be semver matching, as
resolution has already occurred and decided what package versions are
required. Instead, we can just compare the versions directly.
`esbuild` can work fine without needing to run post-install script, so
to make it easier on users (especially people using Vite) we are not prompting to run with
`--allow-scripts` again.
We only do that for version >= 0.18.0 to be sure.
Currently we only warn once. With this PR, we continue to warn about
not-run scripts on explicit `deno install` (or cache). For `run` (or
other subcommands) we only warn the once, as we do currently.
Fixes https://github.com/denoland/deno/issues/25862.
npm only makes bin entries executable if they get linked into `.bin`, as
we did before this PR. So this PR actually deviates from npm, because
it's the only reasonable way to fix this that I can think of.
---
The reason this was broken in moment is the following:
Moment has dependencies on two typescript versions: 1.8 and 3.1
If you have two packages with conflicting bin entries (i.e. two
typescript versions which both have a bin entry `tsc`), in npm it is
non-deterministic and undefined which one will end up in `.bin`.
npm, due to implementation differences, chooses to put typescript 1.8
into the `.bin` directory, and so `node_modules/typescript/bin/tsc` ends
up getting marked executable. We, however, choose typescript 3.2, and so
we end up making `node_modules/typescript3/bin/tsc` executable.
As part of its tests, moment executes `node_modules/typescript/bin/tsc`.
Because we didn't make it executable, this fails.
Since the conflict resolution is undefined in npm, instead of trying to
match it, I think it makes more sense to just make bin entries
executable even if they aren't chosen in the case of a conflict.
Refactors the lifecycle scripts code to extract out the common
functionality and then uses that to provide a warning in the global
resolver.
While ideally we would still support them with the global cache, for now
a warning is at least better than the status quo (where people are
unaware why their packages aren't working).
This was initially added in #25399 in order to make transitioning over
from package.json to deno.json more easy, but it causes some problems
that are shown in the issue and it also means that the output of `deno
install` would have different resolution than `npm install`. Overall, I
think it's too much complexity to be smarter about this and it's
probably best to not do it. If someone needs an aliased folder then they
should keep using a package.json
Closes #25538
This allows using npm deps of jsr deps without having to add them to the
root package.json.
Works by taking the package requirement and scanning the
`node_modules/.deno` directory for the best matching package, so it
relies on deno's node_modules structure.
Additionally to make the transition from package.json to deno.json
easier, Deno now:
1. Installs npm deps in a deno.json at the same time as installing npm
deps from a package.json.
2. Uses the alias in the import map for `node_modules/<alias>` for
better package.json compatiblity.
Stores normalized version constraints in the lockfile, which will
improve reproducibility and will fix a bug with duplicate specifiers
ending up in the lockfile. Also, gets rid of some duplicate data in the
specifiers area of the lockfile.
Regression from
04f9db5b22
Originally I thought to fix the issue in the PR we needed to explicitly
pass through the `node-modules-dir` flag, but after applying the correct
fix that david pointed out (setting `NPM_PROCESS_STATE`) that wasn't
necessary (or correct).
We had a test for deno task with BYONM, but it only tested with
`"unstable": ["byonm"]` in deno.json, so it didn't catch this.
Originally landed in
f6fd6619e7.
Reverted in https://github.com/denoland/deno/pull/24574.
This reland contains a fix that sends "Accept: */*" header for calls made
from "FileFetcher". Absence of this header made downloading source code
from JSR broken. This is tested by ensuring this header is present in the
test server that servers JSR packages.
---------
Co-authored-by: Sean McArthur <sean@seanmonstar.com>
This commit re-implements `ext/fetch` and all dependent crates
using `hyper` and `hyper-util`, instead of `reqwest`.
The reasoning is that we want to have greater control and access
to low level `hyper` APIs when implementing `fetch` API as well
as `node:http` module.
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
NPM inserts a default install script when a package has a `binding.gyp`
file.
It's possible, however, for the package to exclude the `binding.gyp`
file when they publish, and in this case the install script will never
succeed for a user of the package.
This happens with `fsevents`, for instance. They don't include the
`binding.gyp` file in their published tarball, but the default install
script appears in the manifest served by `npm`.
This causes us to warn that `fsevents` has an install script, but when
you try to run it it fails due to `binding.gyp` not existing.
Previously when we printed out the packages that skipped install
scripts, we didn't prefix them with `npm:`. When you pass
`--allow-scripts` though, we require `npm:`, which means you can't just
copy paste the package name from the warning message.
Adds support for running npm package lifecycle scripts, opted into via a
new `--allow-scripts` flag.
With this PR, when running `deno cache` (or `DENO_FUTURE=1 deno
install`) you can specify the `--allow-scripts=pkg1,pkg2` flag to run
lifecycle scripts attached to the given packages.
Note at the moment this only works when `nodeModulesDir` is true (using
the local resolver).
When a package with un-run lifecycle scripts is encountered, we emit a
warning suggesting things may not work and to try running lifecycle
scripts. Additionally, if a package script implicitly requires
`node-gyp` and it's not found on the system, we emit a warning.
Extra things in this PR:
- Extracted out bits of `task.rs` into a separate module for reuse
- Added a couple fields to `process.config` in order to support
`node-gyp` (it relies on a few variables being there)
- Drive by fix to downloading new npm packages to test registry
---
TODO:
- [x] validation for allow-scripts args (make sure it looks like an npm
package)
- [x] make allow-scripts matching smarter
- [ ] figure out what issues this closes
---
Review notes:
- This adds a bunch of deps to our test registry due to using
`node-gyp`, so it's pretty noisy
Adds much better support for the unstable Deno workspaces as well as
support for npm workspaces. npm workspaces is still lacking in that we
only install packages into the root node_modules folder. We'll make it
smarter over time in order for it to figure out when to add node_modules
folders within packages.
This includes a breaking change in config file resolution where we stop
searching for config files on the first found package.json unless it's
in a workspace. For the previous behaviour, the root deno.json needs to
be updated to be a workspace by adding `"workspace":
["./path-to-pkg-json-folder-goes-here"]`. See details in
https://github.com/denoland/deno_config/pull/66
Closes #24340
Closes #24159
Closes #24161
Closes #22020
Closes #18546
Closes #16106
Closes #24160
Closes #18296.
Adds a `--frozen` (alias `--frozen-lockfile`) flag that errors out if
the lockfile is out of date. This is useful for running in CI (where an
out of date lockfile is usually a mistake) or to prevent accidental
changes in dependencies.
As suggested in
https://github.com/denoland/deno/pull/24355#discussion_r1657875422.
I wasn't able to hide the mutex stuff as much as I'd like (ended up just
adding an escape hatch `inner()` method that locks the inner mutex),
because you can't return references to the inner fields through a mutex.
This is mostly motivated by the frozen lockfile changes
