1. There was a lot of cloning going on with `NpmPackageInfo`. This is
now stored in an `Arc<NpmPackageInfo>` and cloning only happens on the
individual version.
2. The package cache is now cleared from memory after resolution.
3. This surfaced a bug in `deno cache` and I noticed it can be more
efficient if we have multiple root specifiers if we provide all the
specifiers as roots.
This PR adds copies of several unstable APIs that are available
in "Deno[Deno.internal].nodeUnstable" namespace.
These copies do not perform unstable check (ie. don't require
"--unstable" flag to be present). Otherwise they work exactly
the same, including permission checks.
These APIs are not meant to be used by users directly and
can change at any time.
Copies of following APIs are available in that namespace:
- Deno.spawnChild
- Deno.spawn
- Deno.spawnSync
- Deno.serve
- Deno.upgradeHttpRaw
- Deno.listenDatagram
This commit makes "npm:" specifiers not require "--unstable" flag.
At the moment some APIs used by Node polyfills still require
"--unstable" which will be addressed in follow up PRs.
This adds support for peer dependencies in npm packages.
1. If not found higher in the tree (ancestor and ancestor siblings),
peer dependencies are resolved like a dependency similar to npm 7.
2. Optional peer dependencies are only resolved if found higher in the
tree.
3. This creates "copy packages" or duplicates of a package when a
package has different resolution due to peer dependency resolution—see
https://pnpm.io/how-peers-are-resolved. Unlike pnpm though, duplicates
of packages will have `_1`, `_2`, etc. added to the end of the package
version in the directory in order to minimize the chance of hitting the
max file path limit on Windows. This is done for both the local
"node_modules" directory and also the global npm cache. The files are
hard linked in this case to reduce hard drive space.
This is a first pass and the code is definitely more inefficient than it
could be.
Closes #15823
This commit fixes CJS resolution when there's a local "node_modules/"
directory.
Before this commit relative imports from CJS files where resolved
relative to
root directory of the package instead of relative to referrer file.
When streaming a resource in ext/http, with compression enabled, we
didn't flush individual chunks. This became very problematic when we
enabled `req.body` from `fetch` for FastStream recently.
This commit now correctly flushes each resource chunk after compression.
This commit adds autodiscovery of lockfile.
This only happens if Deno discovers the configuration file (either
"deno.json" or "deno.jsonc"). In such case Deno tries to load
"deno.lock"
file that sits next to the configuration file, or creates one for user
if
the lockfile doesn't exist yet.
As a consequence, "--lock" and "--lock-write" flags had been updated.
"--lock" no longer requires a value, if one is not provided, it defaults
to "./deno.lock" resolved from the current working directory.
"--lock-write"
description was updated to say that it forces to overwrite a lockfile.
Autodiscovery is currently not handled by the LSP.
In order for test cases to pass regardless of each individual's environment,
this commit adds calls to `slice` method when printing the filenames so
we can avoid getting `console.log` to truncate them.
Fixes #16305
This test has hung a lot recently on macOS. I am not sure if this is
because of a bug in the test or because of the macOS runner that is extremely
slow and flaky in general.
V8's JIT can do a better job knowing the argument count and also enable
fast call path (in future).
This also lets us call async ops without `opAsync`:
```js
const { ops } = Deno.core;
await ops.op_void_async();
```
this patch: 4405286 ops/sec
main: 3508771 ops/sec
This commit stabilizes "Deno.consoleSize()" API.
There is one change compared to previous unstable API,
in that the API doesn't accept any arguments. Console size
is established by querying syscalls for stdio streams at fd
0, 1 and 2.
This commit adds a `reuseAddress` option for UDP sockets. When this
option is enabled, one can listen on an address even though it is
already being listened on from a different process or thread. The new
socket will steal the address from the existing socket.
On Windows and Linux this uses the `SO_REUSEADDR` option, while on other
Unixes this is done with `SO_REUSEPORT`.
This behavior aligns with what libuv does.
TCP sockets still unconditionally set the `SO_REUSEADDR` flag - this
behavior matches Node.js and Go. This PR does not change this behaviour.
Co-authored-by: Luca Casonato <hello@lcas.dev>
When listening on a UNIX socket path, Deno currently tries to unlink
this path prior to actually listening. The implementation of this
behaviour is VERY racy, involves 2 additional syscalls, and does not
match the behaviour of any other runtime (Node.js, Go, Rust, etc).
This commit removes this behaviour. If a user wants to listen on an
existing socket, they must now unlink the file themselves prior to
listening.
This change in behaviour only impacts --unstable APIs, so it is not
a breaking change.
Makes `op_ffi_ptr_of` fast. One of the tests changed from printing
`false` to `true` as the fast `&[u8]` slice path creates the slice with
a null pointer. Thus the `op_ffi_ptr_of` will now return a null pointer
value whereas previously it returned a dangling pointer value.
This PR makes pointer read methods of `Deno.UnsafePointerView` Fast API
compliant, with the exception of `getCString` which cannot be made fast
with current V8 Fast API.
Introduces a new lockfile format that will be used to support locking
"npm" dependencies.
Currently the format looks as follows:
```
// This file is automatically generated by Deno, do not edit its contents
// manually. This file should be commited to your repository.
{
"version": "2",
"remote": {
"https://deno.land/std@0.160.0/http/server.ts": "asdwetsw44523asdfgfas..",
"https://deno.land/std@0.160.0/http/file_server.ts": "asdwetsw44523asdfgfas.."
}
}
```
A follow up PR will add "npm" key that will be used to store information
related
to "npm" dependencies and their resolution.
The new format is used when `--lock-write` is present, if user tries to
load
a lock file using the old format it will still work.
`deno task` has been in use for a few months now. It was very
well received and there are not many complaints. I feel like
this warning might be discouraging for some users and we don't
really plan to make drastic changes to it (besides adding support
for globs in unspecified future).
This commit removes the calls to `expect()` on `std::rc::Rc`, which caused
Deno to panic under certain situations. We now return an error if `Rc`
is referenced by other variables.
Fixes #9360
Fixes #13345
Fixes #13926
Fixes #16241
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This change adds `windowsRawArguments` to `SpawnOptions`. The option enables
skipping the default quoting and escaping while creating the command on
windows.
The option works in a similar way as `windowsVerbatimArguments` in
child_process.spawn options in Node.js, and is necessary for simulating
it in `std/node`.
closes #8852
This PR fixes invalid header parsing which is flaky because `g` flag is
being used in the regex, which keeps track of `lastIndex`
```javascript
try {
new Headers([["x", "\u0000x"]]); // error
} catch(e) {}
new Headers([["x", "\u0000x"]]); // no error
```
This issue affects `Response` & `Request` constructors as well
This commit introduces two new buffer wrapper types to `deno_core`. The
main benefit of these new wrappers is that they can wrap a number of
different underlying buffer types. This allows for a more flexible read
and write API on resources that will require less copying of data
between different buffer representations.
- `BufView` is a read-only view onto a buffer. It can be backed by
`ZeroCopyBuf`, `Vec<u8>`, and `bytes::Bytes`.
- `BufViewMut` is a read-write view onto a buffer. It can be cheaply
converted into a `BufView`. It can be backed by `ZeroCopyBuf` or
`Vec<u8>`.
Both new buffer views have a cursor. This means that the start point of
the view can be constrained to write / read from just a slice of the
view. Only the start point of the slice can be adjusted. The end point
is fixed. To adjust the end point, the underlying buffer needs to be
truncated.
Readable resources have been changed to better cater to resources that
do not support BYOB reads. The basic `read` method now returns a
`BufView` instead of taking a `ZeroCopyBuf` to fill. This allows the
operation to return buffers that the resource has already allocated,
instead of forcing the caller to allocate the buffer. BYOB reads are
still very useful for resources that support them, so a new `read_byob`
method has been added that takes a `BufViewMut` to fill. `op_read`
attempts to use `read_byob` if the resource supports it, which falls
back to `read` and performs an additional copy if it does not. For
Rust->JS reads this change should have no impact, but for Rust->Rust
reads, this allows the caller to avoid an additional copy in many
scenarios. This combined with the support for `BufView` to be backed by
`bytes::Bytes` allows us to avoid one data copy when piping from a
`fetch` response into an `ext/http` response.
Writable resources have been changed to take a `BufView` instead of a
`ZeroCopyBuf` as an argument. This allows for less copying of data in
certain scenarios, as described above. Additionally a new
`Resource::write_all` method has been added that takes a `BufView` and
continually attempts to write the resource until the entire buffer has
been written. Certain resources like files can override this method to
provide a more efficient `write_all` implementation.
Currently Content-Length is sent when the status code is 204. However,
according to the spec, this should not be sent.
Modify the if statement below to prevent the Content-Length from being
sent.
This commit adds a fast path to `Request` and `Response` that
make consuming request bodies much faster when using `Body#text`,
`Body#arrayBuffer`, and `Body#blob`, if the body is a FastStream.
Because the response bodies for `fetch` are FastStream, this speeds up
consuming `fetch` response bodies significantly.
This commit adds support for npm specifier in "deno cache" subcommand.
```
$ deno cache --unstable npm:vite npm:chalk https://deno.land/std/http/file_server.ts
```
Besides downloading requested npm package(s), it will also download
necessary code from "std/node/".
Make offering "virtual documents" via the lsp easier to parse. `deno:`
can be ambiguous to parse by editors (can conflict with linux paths)
Neovim recently landed a PR https://github.com/neovim/neovim/pull/19797
that allows it to parse `scheme:/` this PR should make deno lsp work
correctly in neovim
This commit allows the Node compatibility layer to skip
environment variable permission checks when --unstable
is passed and the variable name is one that Node uses.
Fixes: https://github.com/denoland/deno/issues/15890
This commit makes error objects more resistant to
prototype tampering.
This bug was found when updating the deno_std Node compatibility
layer to Node 18. The Node test 'parallel/test-assert-fail.js'
was breaking std's assertion library.
Refs: https://github.com/denoland/deno_std/pull/2585
This commit removes "compat" mode. We shipped support for "npm:" specifier
support in v1.25 and that is preferred way to interact with Node code that we
will iterate and improve upon.
Previously `jsxImportSource` was resolved relative to the config file
during graph building, and relative to the emitted module during
runtime.
This is now fixed so that the JSX import source is resolved relative to
the module both during graph building and at runtime.
This commit splits `Deno.upgradeHttp` into two different APIs, because
the same API is currently overloaded with two different functions. Flash
requests upgrade immediately, with no need to return a `Response`
object. Instead you have to manually write the response to the socket.
Hyper requests only upgrade once a `Response` object has been sent.
These two behaviours are now split into `Deno.upgradeHttp` and
`Deno.upgradeHttpRaw`. The latter is flash only. The former only
supports hyper requests at the moment, but can be updated to support
flash in the future.
Additionally this removes `void | Promise<void>` as valid return types
for the handler function. If one wants to use `Deno.upgradeHttpRaw`,
they will have to type cast the handler signature - the signature is
meant for the 99.99%, and should not be complicated for the 0.01% that
use `Deno.upgradeHttpRaw()`.
This commit changes the `Deno.serve` function signature to be more
versatile and easier to use. It is now a drop in replacement for
std/http's `serve`.
The input validation has also been reworked.
Previously if a user specified a content-length header for an POST
request without a body, the request would contain two `content-length`
headers. One added by us, and one added by the user.
This commit ignores all content-length headers coming from the user,
because we need to have the sole authority on the content-length because
we transmit the body.
This commit changes "npm:" specifier handling to respect "--cached-only" flags and adds "Download" messages for npm registry api calls.
Co-authored-by: David Sherret <dsherret@gmail.com>
Adds error event dispatching for queueMicrotask(). Consequently unhandled errors are now reported with Deno.core.terminate(), which is immune to the existing quirk with plainly thrown errors (#14158).
This adds an init subcommand to that creates a project starter similar to cargo init.
```
$ deno init my_project
Project initialized
Run these commands to get started:
cd my_project
deno run main.ts
deno run main_test.ts
$ deno run main.ts
Add 2 + 3 5
$ cat main.ts
export function add(a: number, b: number): number {
return a + b;
}
if (import.meta.main) {
console.log("Add 2 + 3", add(2, 3));
}
$ cat main_test.ts
import { assertEquals } from "https://deno.land/std@0.151.0/testing/asserts.ts";
import { add } from "./main.ts";
Deno.test(function addTest() {
assertEquals(add(2, 3), 5);
});
```
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
- Merge "Deno.serve()" and "Deno.serveTls()" API
- Remove first argument and use "fetch" field options instead
- Update type declarations
- Add more documentation
Welcome to better optimised op calls! Currently opSync is called with parameters of every type and count. This most definitely makes the call megamorphic. Additionally, it seems that spread params leads to V8 not being able to optimise the calls quite as well (apparently Fast Calls cannot be used with spread params).
Monomorphising op calls should lead to some improved performance. Now that unwrapping of sync ops results is done on Rust side, this is pretty simple:
```
opSync("op_foo", param1, param2);
// -> turns to
ops.op_foo(param1, param2);
```
This means sync op calls are now just directly calling the native binding function. When V8 Fast API Calls are enabled, this will enable those to be called on the optimised path.
Monomorphising async ops likely requires using callbacks and is left as an exercise to the reader.
This commit adds "ext/node" extension that implementes CommonJS module system.
In the future this extension might be extended to actually contain implementation of
Node compatibility layer in favor of "deno_std/node".
Currently this functionality is not publicly exposed, it is available via "Deno[Deno.internal].require"
namespace and is meant to be used by other functionality to be landed soon.
This is a minimal first pass, things that still don't work:
support for dynamic imports in CJS
conditional exports
This commit adds new "import.meta.resolve()" API which
allows to resolve specifiers relative to the module the API
is called in. This API supports resolving using import maps.
- "SpawnOutput" extends "ChildStatus" instead of composing it
- "SpawnOutput::stdout", "SpawnOutput::stderr", "Child::stdin",
"Child::stdout" and "Child::stderr" are no longer optional, instead
made them getters that throw at runtime if that stream wasn't set
to "piped".
- Remove the complicated "<T extends SpawnOptions = SpawnOptions>"
which we currently need to give proper type hints for the availability of
these fields. Their typings for these would get increasingly complex
if it became dependent on more options (e.g. "SpawnOptions::pty"
which if set should make the stdio streams unavailable)
Relanding #12994
This commit adds support for "unhandledrejection" event.
This event will trigger event listeners registered using:
"globalThis.addEventListener("unhandledrejection")
"globalThis.onunhandledrejection"
This is done by registering a default handler using
"Deno.core.setPromiseRejectCallback" that allows to
handle rejected promises in JavaScript instead of Rust.
This commit will make it possible to polyfill
"process.on("unhandledRejection")" in the Node compat
layer.
Co-authored-by: Colin Ihrig <cjihrig@gmail.com>
This commit adds support for "unhandledrejection" event.
This event will trigger event listeners registered using:
"globalThis.addEventListener("unhandledrejection")
"globalThis.onunhandledrejection"
This is done by registering a default handler using
"Deno.core.setPromiseRejectCallback" that allows to
handle rejected promises in JavaScript instead of Rust.
This commit will make it possible to polyfill
"process.on("unhandledRejection")" in the Node compat
layer.
Co-authored-by: Colin Ihrig <cjihrig@gmail.com>
When a dynamically imported module gets resolved, any code that comes after an
await import() to that module will continue running. However, if that is the
last code in the evaluation of another dynamically imported module, that second
module will not resolve until the next iteration of the event loop, even though
it does not depend on the event loop at all.
When the event loop is being blocked by a long-running operation, such as a
long-running timer, or by an async op that might never end, such as with workers
or BroadcastChannels, that will result in the second dynamically imported module
not being resolved for a while, or ever.
This change fixes this by running the dynamic module loading steps in a loop
until no more dynamic modules can be resolved.
This commit adds support for unstable FFI
callbacks. A callback is registered using
the `Deno.UnsafeCallback` API.
The backing memory for the callback can
be disposed of using `Deno.UnsafeCallback#close`.
It is not safe to pass the callback after calling
close.
Callbacks from other than the isolate thread
are not supported.
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Co-authored-by: Bert Belder <bertbelder@gmail.com>
This commit adds brand checking to EventTarget. It also fixes a
bug where deno would crash if an abort signal was aborted on the
global addEventListener().
This commit updates the custom inspect function for URL objects
to pass the inspect options through so that the context is
propagated and the resulting indentation is correct.
Fixes: https://github.com/denoland/deno/issues/14171
This commit changes default default behavior of type checking
for several subcommands.
Instead of type checking and reporting type errors only for local
files, the type checking is skipped entirely. Type checking can
still be enabled using the "--check" flag.
Following subcomands are affected:
- deno cache
- deno install
- deno eval
- deno run
This commit adds "Deno.core.setFormatExceptionCallback" which
can be used to provide custom formatting for errors. It is useful
in cases when user throws something that is non-Error (eg.
a string, plain object, etc).
This commit improves the permission descriptor validation by
explicitly checking for object types and using optional
chaining when creating error messages in case the descriptor
is not an object.
Fixes: https://github.com/denoland/deno/issues/14675
This commit changes default mode of type-checking to "local"
and adds "--check" flag to following subcommands:
- deno bench
- deno bundle
- deno cache
- deno compile
- deno eval
- deno install
- deno test
This commit removes "WorkerOptions.deno" option as a boolean,
as well as "WorkerOptions.deno.namespace" settings. Starting
with this commit all workers have access to "Deno" namespace
by default.
This commit fixes source maps for files that contain emojis.
This is done by updating "deno_ast" to "0.14.1" for the case
of "--no-check" flag (ie using SWC emit) and by overriding
TSC's default base64 encoder (which turned out to be buggy)
for the type checking case.
This flag disables loading of configuration file, ie. it will not be
automatically discovered and loaded. Of course this flag conflicts
with "--config" flag and they cannot be used together.
This commit adds better reporting of uncaught errors
in top level scope of testing files. This change affects
both console runner as well as LSP runner.
Calling `worker.terminate()` used to kill the worker's isolate and
then block until the worker's thread finished. This blocks the calling
thread if the worker's event loop was blocked in a sync op (as with
`Deno.sleepSync`), which wasn't realized at the time, but since the
worker's isolate was killed at that moment, it would not block the
calling thread if the worker was in a JS endless loop.
However, in #12831, in order to work around a V8 bug, worker
termination was changed to first set a signal to let the worker event
loop know that termination has been requested, and only kill the
isolate if the event loop has not finished after 2 seconds. However,
this change kept the blocking, which meant that JS endless loops in
the worker now blocked the parent for 2 seconds.
As it turns out, after #12831 it is fine to signal termination and
even kill the worker's isolate without waiting for the thread to
finish, so this change does that. However, that might leave the async
ops that receive messages and control data from the worker pending
after `worker.terminate()`, which leads to odd results from the op
sanitizer. Therefore, we set up a `CancelHandler` to cancel those ops
when the worker is terminated.
This commit:
- removes "fmt_errors::PrettyJsError" in favor of "format_js_error" fn
- removes "deno_core::JsError::create" and
"deno_core::RuntimeOptions::js_error_create_fn"
- adds new option to "deno_runtime::ops::worker_host::init"
stream shutdown wasn't happening correctly (moved it to call op_http_shutdown) & extra zeroed bytes were being sent for when body length not a multiple of 64*1024
