dd8cbf5e29
Fixes #22995. Fixes #23000. There were a handful of bugs here causing the hang (each with a corresponding minimized test): - We were canceling recv futures when `receiveMessageOnPort` was called, but this caused the "receive loop" in the message port to exit. This was due to the fact that `CancelHandle`s are never reset (i.e., once you `cancel` a `CancelHandle`, it remains cancelled). That meant that after `receieveMessageOnPort` was called, the subsequent calls to `op_message_port_recv_message` would throw `Interrupted` exceptions, and we would exit the loop. The cancellation, however, isn't actually necessary. `op_message_port_recv_message` only borrows the underlying port for long enough to poll the receiver, so the borrow there could never overlap with `op_message_port_recv_message_sync`. - Calling `MessagePort.unref()` caused the "receive loop" in the message port to exit. This was because we were setting `messageEventListenerCount` to 0 on unref. Not only does that break the counter when multiple `MessagePort`s are present in the same thread, but we also exited the "receive loop" whenever the listener count was 0. I assume this was to prevent the recv promise from keeping the event loop open. Instead of this, I chose to just unref the recv promise as needed to control the event loop. - The last bug causing the hang (which was a doozy to debug) ended up being an unfortunate interaction between how we implement our messageport "receive loop" and a pattern found in `npm:piscina` (which angular uses). The gist of it is that piscina uses an atomic wait loop along with `receiveMessageOnPort` in its worker threads, and as the worker is getting started, the following incredibly convoluted series of events occurs: 1. Parent sends a MessagePort `p` to worker 2. Parent sends a message `m` to the port `p` 3. Parent notifies the worker with `Atomics.notify` that a new message is available 4. Worker receives message, adds "message" listener to port `p` 5. Adding the listener triggers `MessagePort.start()` on `p` 6. Receive loop in MessagePort.start receives the message `m`, but then hits an await point and yields (before dispatching the "message" event) 7. Worker continues execution, starts the atomic wait loop, and immediately receives the existing notification from the parent that a message is available 8. Worker attempts to receive the new message `m` with `receiveMessageOnPort`, but this returns `undefined` because the receive loop already took the message in 6 9. Atomic wait loop continues to next iteration, waiting for the next message with `Atomic.wait` 10. `Atomic.wait` blocks the worker thread, which prevents the receive loop from continuing and dispatching the "message" event for the received message 11. The parent waits for the worker to respond to the first message, and waits 12. The thread can't make any more progress, and the whole process hangs The fix I've chosen here (which I don't particularly love, but it works) is to just delay the `MessagePort.start` call until the end of the event loop turn, so that the atomic wait loop receives the message first. This prevents the hang. --- Those were the main issues causing the hang. There ended up being a few other small bugs as well, namely `exit` being emitted multiple times, and not patching up the message port when it's received by `receiveMessageOnPort`. |
||
---|---|---|
.. | ||
_fs | ||
_process | ||
_util | ||
assert | ||
dns | ||
fs | ||
internal | ||
internal_binding | ||
path | ||
readline | ||
stream | ||
timers | ||
util | ||
00_globals.js | ||
01_require.js | ||
02_init.js | ||
_brotli.js | ||
_events.d.ts | ||
_events.mjs | ||
_global.d.ts | ||
_http_agent.mjs | ||
_http_common.ts | ||
_http_outgoing.ts | ||
_http_server.ts | ||
_next_tick.ts | ||
_readline.d.ts | ||
_readline.mjs | ||
_readline_shared_types.d.ts | ||
_stream.d.ts | ||
_stream.mjs | ||
_tls_common.ts | ||
_tls_wrap.ts | ||
_utils.ts | ||
_zlib.mjs | ||
_zlib_binding.mjs | ||
assert.ts | ||
assertion_error.ts | ||
async_hooks.ts | ||
buffer.ts | ||
child_process.ts | ||
cluster.ts | ||
console.ts | ||
constants.ts | ||
crypto.ts | ||
dgram.ts | ||
diagnostics_channel.js | ||
dns.ts | ||
domain.ts | ||
events.ts | ||
fs.ts | ||
http.ts | ||
http2.ts | ||
https.ts | ||
inspector.ts | ||
net.ts | ||
os.ts | ||
path.ts | ||
perf_hooks.ts | ||
process.ts | ||
punycode.ts | ||
querystring.js | ||
readline.ts | ||
README.md | ||
repl.ts | ||
stream.ts | ||
string_decoder.ts | ||
sys.ts | ||
testing.ts | ||
timers.ts | ||
tls.ts | ||
trace_events.ts | ||
tty.js | ||
url.ts | ||
util.ts | ||
v8.ts | ||
vm.js | ||
wasi.ts | ||
worker_threads.ts | ||
zlib.ts |
Deno Node.js compatibility
This module is meant to have a compatibility layer for the Node.js standard library.
Warning: Any function of this module should not be referred anywhere in the Deno standard library as it's a compatibility module.
Supported modules
- assert
- assert/strict partly
- async_hooks partly
- buffer
- child_process partly
- cluster partly
- console partly
- constants partly
- crypto partly
- dgram partly
- diagnostics_channel partly
- dns partly
- events
- fs partly
- fs/promises partly
- http partly
- http2
- https partly
- inspector partly
- module
- net
- os partly
- path
- path/posix
- path/win32
- perf_hooks
- process partly
- punycode
- querystring
- readline
- repl partly
- stream
- stream/promises
- stream/web partly
- string_decoder
- sys
- timers
- timers/promises
- tls
- trace_events
- tty partly
- url
- util partly
- util/types partly
- v8
- vm partly
- wasi
- webcrypto
- worker_threads
- zlib
- node globals partly
Deprecated
These modules are deprecated in Node.js and will probably not be polyfilled:
- domain
- freelist
Experimental
These modules are experimental in Node.js and will not be polyfilled until they are stable:
- diagnostics_channel
- async_hooks
- policies
- trace_events
- wasi
- webcrypto
CommonJS modules loading
createRequire(...)
is provided to create a require
function for loading CJS
modules. It also sets supported globals.
import { createRequire } from "node:module";
const require = createRequire(import.meta.url);
// Loads native module polyfill.
const path = require("path");
// Loads extensionless module.
const cjsModule = require("./my_mod");
// Visits node_modules.
const leftPad = require("left-pad");
Contributing
Setting up the test runner and running tests
See tests/node_compat/runner/README.md.
Best practices
When converting from promise-based to callback-based APIs, the most obvious way is like this:
promise.then((value) => callback(null, value)).catch(callback);
This has a subtle bug - if the callback throws an error, the catch statement will also catch that error, and the callback will be called twice. The correct way to do it is like this:
promise.then((value) => callback(null, value), callback);
The second parameter of then
can also be used to catch errors, but only errors
from the existing promise, not the new one created by the callback.
If the Deno equivalent is actually synchronous, there's a similar problem with try/catch statements:
try {
const value = process();
callback(null, value);
} catch (err) {
callback(err);
}
Since the callback is called within the try
block, any errors from it will be
caught and call the callback again.
The correct way to do it is like this:
let err, value;
try {
value = process();
} catch (e) {
err = e;
}
if (err) {
callback(err); // Make sure arguments.length === 1
} else {
callback(null, value);
}
It's not as clean, but prevents the callback being called twice.
Remaining Tests
Node compatibility can be measured by how many native Node tests pass. If you'd like to know what you can work on, check out the list of Node tests remaining here.