1
0
Fork 0
mirror of https://github.com/denoland/deno.git synced 2024-12-11 01:58:05 -05:00
Commit graph

56 commits

Author SHA1 Message Date
Bartek Iwańczuk
797f0ef42e
fix(npm): canonicalize filename before returning (#18948)
This commit changes how paths for npm packages are handled,
by canonicalizing them when resolving. This is done so that instead
of returning
"node_modules/<package_name>@<version>/node_modules/<dep>/index.js"
(which is a symlink) we "node_modules/<dep>@<dep_version>/index.js.

Fixes https://github.com/denoland/deno/issues/18924
Fixes https://github.com/bluwy/create-vite-extra/issues/31

---------

Co-authored-by: David Sherret <dsherret@gmail.com>
2023-05-04 16:18:57 +02:00
David Sherret
5b4a9b48ae
refactor(ext/node): enforce interior mutable for NodePermissions to remove clones (#18831)
We can make `NodePermissions` rely on interior mutability (which the
`PermissionsContainer` is already doing) in order to not have to clone
everything all the time. This also reduces the chance of an accidental
`borrow` while `borrrow_mut`.
2023-04-24 21:07:48 -04:00
David Sherret
aa286fdecb
refactor(ext/node): allow injecting NodeFs from CLI (#18829)
This allows providing a `NodeFs` as part of the `WorkerOptions`.
2023-04-24 19:44:35 -04:00
David Sherret
0e97fa4d5f
fix(npm): only include top level packages in top level node_modules directory (#18824)
We were indeterministically including packages in the top level
`node_modules/` folder when using a local node_modules directory. This
change aligns with pnpm and only includes top level packages in this
folder. This should be faster for initializing the folder, but may
expose issues in packages that reference other packages not defined in
their dependencies. That said, the behaviour previously was previously
broken.

This has exposed a bug in the require implementation where it doesn't
find a package (which is the main underlying issue here). There is a
failing test already for this in the test suite after this change.

Closes #18822

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-04-24 17:08:11 -04:00
David Sherret
a615eb3b56
refactor(node): move most of cli/node to ext/node (#18797)
This is just a straight refactor and I didn't do any cleanup in
ext/node. After this PR we can start to clean it up and make things
private that don't need to be public anymore.
2023-04-21 21:02:46 -04:00
David Sherret
4a33c349af
refactor: move some CJS and ESM code analysis to ext/node (#18789) 2023-04-21 16:38:10 -04:00
David Sherret
d2d62b6312
refactor(npm): add CliNodeResolver (#18742) 2023-04-17 15:36:23 -04:00
David Sherret
136dce67ce
refactor: break up ProcState (#18707)
1. Breaks up functionality within `ProcState` into several other structs
to break out the responsibilities (`ProcState` is only a data struct
now).
2. Moves towards being able to inject dependencies more easily and have
functionality only require what it needs.
3. Exposes `Arc<T>` around the "service structs" instead of it being
embedded within them. The idea behind embedding them was to reduce the
verbosity of needing to pass around `Arc<...>`, but I don't think it was
exactly working and as we move more of these structs to be more
injectable I don't think the extra verbosity will be a big deal.
2023-04-14 16:22:33 -04:00
David Sherret
efa7c19890
refactor: upgrade to deno_npm 0.3.0 (#18671)
This allows us to specify the `@types/node` version constraint in the
CLI instead of in deno_npm.
2023-04-13 10:47:45 -04:00
David Sherret
5c7f76c570
fix(npm): reload an npm package's dependency's information when version not found (#18622)
This reloads an npm package's dependency's information when a
version/version req/tag is not found.

This PR applies only to dependencies of npm packages. It does NOT yet
cause npm specifiers to have their dependency information cache busted.
That requires a different solution, but this should help cache bust in
more scenarios.

Part of #16901, but doesn't close it yet
2023-04-06 21:41:19 -04:00
David Sherret
d07aa4a072
refactor(npm): use deno_npm and deno_semver (#18602) 2023-04-06 18:46:44 -04:00
Geert-Jan Zwiers
206c593519
fix(coverage): ignore files from npm registry (#18457)
Fixes https://github.com/denoland/deno/issues/17664 and part of
https://github.com/denoland/deno/issues/18454 by excluding files
belonging to npm modules by default in the coverage output.
2023-03-30 17:40:22 +00:00
Bartek Iwańczuk
913e2875c1
refactor(ext/node): add NodeEnv::Fs associated type (#18484)
This commit adds associated type to "NodeEnv" trait, called "Fs".

The "Fs" type has a trait bound on "NodeFs", which specifies APIs
required for all ops and resolution APIs to function.

A "RealFs" implementation of "NodeFs" is exported from the "deno_node"
crate, that provides a default implementation for the trait.

All code in "deno_node" extension was changed to use the "NodeFs" trait
to handle file system operations, instead of relying on APIs from the
standard library.
2023-03-30 03:20:31 +02:00
David Sherret
44b0d4cb11
fix(npm): show a progress bar when initializing the node_modules folder (#18136)
Creating the node_modules folder when the packages are already
downloaded can take a bit of time and not knowing what is going on can
be confusing. It's better to show a progress bar.
2023-03-13 18:18:29 +00:00
David Sherret
11f225ef7d
fix: ensure no node_modules directory is created when a package.json exists and no npm dependencies are used (#18134)
Closes #18133
Closes #18038
2023-03-13 14:03:19 +00:00
David Sherret
bcb6ee9d08
refactor(npm): push npm struct creation to a higher level (#18139)
This has been bothering me for a while and it became more painful while
working on #18136 because injecting the shared progress bar became very
verbose. Basically we should move the creation of all these npm structs
up to a higher level.

This is a stepping stone for a future refactor where we can improve how
we create all our structs.
2023-03-12 23:32:59 -04:00
David Sherret
88b5fd9088
fix: attempt to only allow one deno process to update the node_modules folder at a time (#18058)
This is implemented in such a way that it should still allow processes
to go through when a file lock wasn't properly cleaned up and the OS
hasn't released it yet (but with a 200ms-ish delay).

Closes #18039
2023-03-08 15:13:13 +00:00
David Sherret
033b70af19
fix(npm): lazily install package.json dependencies only when necessary (#17931)
This lazily does an "npm install" when any package name matches what's
found in the package.json or when running a script from package.json
with deno task.

Part of #17916

Closes #17928
2023-02-24 19:35:43 -05:00
David Sherret
6233c0aff0
fix(npm): support bare specifiers in package.json having a path (#17903)
For example `import * as test from "package/path.js"`
2023-02-23 17:33:23 +00:00
David Sherret
344317ec50
feat(npm): support bare specifiers from package.json in more subcommands and language server (#17891) 2023-02-23 10:58:10 -05:00
David Sherret
b15f9e60a0
feat(task): support scripts in package.json (#17887)
This is a super basic initial implementation. We don't create a
`node_modules/.bin` folder at the moment and add it to the PATH like we
should which is necessary to make command name resolution in the
subprocess work properly (ex. you run a script that launches another
script that then tries to launch an "npx command"... this won't work
atm).

Closes #17492
2023-02-22 22:45:35 -05:00
David Sherret
a6ca4d0d61
refactor: use deno_graph for npm specifiers (#17858)
This changes npm specifiers to be handled by deno_graph and resolved to
an npm package name and version when the specifier is encountered. It
also slightly changes how npm specifier resolution occurs—previously it
would collect all the npm specifiers and resolve them all at once, but
now it resolves them on the fly as they are encountered in the module
graph.

https://github.com/denoland/deno_graph/pull/232

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-02-22 14:15:25 -05:00
David Sherret
3479bc7661
fix(npm): improve peer dependency resolution (#17835)
This PR fixes peer dependency resolution to only resolve peers based on
the current graph traversal path. Previously, it would resolve a peers
by looking at a graph node's ancestors, which is not correct because
graph nodes are shared by different resolutions.

It also stores more information about peer dependency resolution in the
lockfile.
2023-02-21 12:03:48 -05:00
Bartek Iwańczuk
4d1a14ca7f
feat: auto-discover package.json for npm dependencies (#17272)
This commits adds auto-discovery of "package.json" file when running
"deno run" and "deno task" subcommands. In case of "deno run" the
"package.json" is being looked up starting from the directory of the
script that is being run, stopping early if "deno.json(c)" file is found
(ie. FS tree won't be traversed "up" from "deno.json").

When "package.json" is discovered the "--node-modules-dir" flag is
implied, leading to creation of local "node_modules/" directory - we
did that, because most tools relying on "package.json" will expect
"node_modules/" directory to be present (eg. Vite). Additionally 
"dependencies" and "devDependencies" specified in the "package.json"
are downloaded on startup. 

This is a stepping stone to supporting bare specifier imports, but
the actual integration will be done in a follow up commit.

---------

Co-authored-by: David Sherret <dsherret@gmail.com>
2023-02-20 19:14:06 +01:00
David Sherret
610b8cc2bf
refactor: add NpmPackageId back from deno_graph as NpmPackageNodeId (#17804)
The `NpmPackageId` struct is being renamed to `NpmPackageNodeId`. In a
future PR it will be moved down into only npm dependency resolution and
a `NpmPackageId` struct will be introduced in `deno_graph` that only has
the name and version of the package (no peer dependency identifier
information). So a `NpmPackageReq` will map to an `NpmPackageId`, which
will map to an `NpmPackageNodeId` in the npm resolution.
2023-02-17 09:12:22 -05:00
David Sherret
c4b9a91e27
refactor: use deno_graph's semver and npm structs (#17791) 2023-02-15 13:20:40 -05:00
David Sherret
f5840bdcd3
chore: upgrade to Rust 1.67 (#17548)
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2023-01-27 10:43:16 -05:00
Bartek Iwańczuk
fc2e00152b
feat: support node built-in module imports (#17264)
Co-authored-by: David Sherret <dsherret@gmail.com>
2023-01-24 09:05:54 -05:00
David Sherret
40527526e5
refactor(npm): avoid cloning snapshot for lockfile (#17451) 2023-01-16 19:46:30 +00:00
Bartek Iwańczuk
636352e0ca
fix(npm): allow to read package.json if permissions are granted (#17209)
This commit changes signature of "deno_core::ModuleLoader::resolve" to pass
an enum indicating whether or not we're resolving a specifier for dynamic import.

Additionally "CliModuleLoader" was changes to store both "parent permissions" (or
"root permissions") as well as "dynamic permissions" that allow to check for permissions
in top-level module load an dynamic imports.

Then all code paths that have anything to do with Node/npm compat are now checking
for permissions which are passed from module loader instance associated with given
worker.
2023-01-10 14:35:44 +01:00
David Sherret
10e4b2e140
chore: update copyright year to 2023 (#17247)
Yearly tradition of creating extra noise in git.
2023-01-02 21:00:42 +00:00
Ikko Ashimine
1a3665b612
chore(npm): fix typo in local.rs (#17186) 2022-12-26 17:24:25 +09:00
David Sherret
54d40e008a
perf(lsp): concurrent reads and exclusive writes (#17135) 2022-12-19 20:22:17 -05:00
linbingquan
f46df3e359
chore: update to Rust 1.66.0 (#17078) 2022-12-17 23:20:15 +01:00
David Sherret
3973ceb634
fix(npm): dependency types were sometimes not being resolved when package had no types entry (#16958)
Closes #16957
2022-12-05 20:09:31 -05:00
David Sherret
b1e29d1bd0
fix(npm): improve package.json exports support for types (#16880) 2022-11-30 23:07:32 +00:00
David Sherret
2d4c46c975
refactor: create util folder, move nap_sym to napi/sym, move http_cache to cache folder (#16857) 2022-11-28 17:28:54 -05:00
David Sherret
fb04e87387
fix(npm): ensure npm package downloaded once per run when using --reload (#16842) 2022-11-27 13:25:08 -05:00
David Sherret
0cc90d9246
refactor: move lockfile.rs to args module (#16818)
This should be in the `args` folder as it's similar to `config_file`.
2022-11-25 17:00:28 -05:00
Aaron O'Mullan
238590aa9f
chore: use Rust 1.65.0 (#16688) 2022-11-18 02:59:10 +01:00
Bartek Iwańczuk
bd159b8bad
fix(lock): ensure npm dependencies are written with --lock-write (#16668)
If "--lock-write" flag was present we never passed instance of the lockfile to 
the npm resolver, which made it skip adding discovered npm packages to
the lockfile. This commit fixes that, by always passing lockfile to the npm
resolver and only regenerating resolver snapshot is "--lock-write" is not
present.

Closes https://github.com/denoland/deno/issues/16666
2022-11-16 19:59:26 +01:00
David Sherret
40a72f3555
fix(npm): support non-all lowercase package names (#16669)
Supports package names that aren't all lowercase.

This stores the package with a leading underscore (since that's not
allowed in npm's registry and no package exists with a leading
underscore) then base32 encoded (A-Z0-9) so it can be lowercased and
avoid collisions.

Global cache dir:

```
$DENO_DIR/npm/registry.npmjs.org/_{base32_encode(package_name).to_lowercase()}/{version}
```

node_modules dir `.deno` folder:

```
node_modules/.deno/_{base32_encode(package_name).to_lowercase()}@{version}/node_modules/<package-name>
```

Within node_modules folder:

```
node_modules/<package-name>
```

So, direct childs of the node_modules folder can have collisions between
packages like `JSON` vs `json`, but this is already something npm itself
doesn't handle well. Plus, Deno doesn't actually ever resolve to the
`node_modules/<package-name>` folder, but just has that for
compatibility. Additionally, packages in the `.deno` dir could have
collissions if they have multiple dependencies that only differ in
casing or a dependency that has different casing, but if someone is
doing that then they're already going to have trouble with npm and they
are asking for trouble in general.
2022-11-16 18:44:31 +00:00
David Sherret
38542e849d
fix(npm): properly handle getting @types package for scoped packages (#16655) 2022-11-15 21:48:29 -05:00
David Sherret
d81065cff9
feat(unstable/npm): module graph derived npm specifier resolution order (#16602) 2022-11-11 21:26:14 -05:00
Bartek Iwańczuk
53e974b276
feat: remove --unstable flag requirement for npm: specifiers (#16473)
This commit makes "npm:" specifiers not require "--unstable" flag.
At the moment some APIs used by Node polyfills still require
"--unstable" which will be addressed in follow up PRs.
2022-11-10 17:57:10 +01:00
David Sherret
cbb3f85433
feat(unstable/npm): support peer dependencies (#16561)
This adds support for peer dependencies in npm packages.

1. If not found higher in the tree (ancestor and ancestor siblings),
peer dependencies are resolved like a dependency similar to npm 7.
2. Optional peer dependencies are only resolved if found higher in the
tree.
3. This creates "copy packages" or duplicates of a package when a
package has different resolution due to peer dependency resolution—see
https://pnpm.io/how-peers-are-resolved. Unlike pnpm though, duplicates
of packages will have `_1`, `_2`, etc. added to the end of the package
version in the directory in order to minimize the chance of hitting the
max file path limit on Windows. This is done for both the local
"node_modules" directory and also the global npm cache. The files are
hard linked in this case to reduce hard drive space.

This is a first pass and the code is definitely more inefficient than it
could be.

Closes #15823
2022-11-08 14:17:24 -05:00
David Sherret
edaceecec7
feat: support npm specifiers in deno info for display text output only (#16470) 2022-10-28 16:19:55 -04:00
Bartek Iwańczuk
9835b095e5
fix(npm): add support for npm packages in lock files (#15938)
This commit adds support for npm packages in the lock file.
2022-10-25 18:20:07 +02:00
David Sherret
bcfe279fba
feat(unstable/npm): initial type checking of npm specifiers (#16332) 2022-10-21 15:20:18 +00:00
David Sherret
9df8d9d831
perf(npm): parallelize caching of npm specifier package infos (#16323) 2022-10-17 12:27:31 -04:00