1
0
Fork 0
mirror of https://github.com/denoland/deno.git synced 2024-11-24 15:19:26 -05:00
denoland-deno/ext/web
Nathan Whitaker dd8cbf5e29
fix(node): fix worker_threads issues blocking Angular support (#26024)
Fixes #22995. Fixes #23000.

There were a handful of bugs here causing the hang (each with a
corresponding minimized test):

- We were canceling recv futures when `receiveMessageOnPort` was called,
but this caused the "receive loop" in the message port to exit. This was
due to the fact that `CancelHandle`s are never reset (i.e., once you
`cancel` a `CancelHandle`, it remains cancelled). That meant that after
`receieveMessageOnPort` was called, the subsequent calls to
`op_message_port_recv_message` would throw `Interrupted` exceptions, and
we would exit the loop.

The cancellation, however, isn't actually necessary.
`op_message_port_recv_message` only borrows the underlying port for long
enough to poll the receiver, so the borrow there could never overlap
with `op_message_port_recv_message_sync`.

- Calling `MessagePort.unref()` caused the "receive loop" in the message
port to exit. This was because we were setting
`messageEventListenerCount` to 0 on unref. Not only does that break the
counter when multiple `MessagePort`s are present in the same thread, but
we also exited the "receive loop" whenever the listener count was 0. I
assume this was to prevent the recv promise from keeping the event loop
open.

Instead of this, I chose to just unref the recv promise as needed to
control the event loop.

- The last bug causing the hang (which was a doozy to debug) ended up
being an unfortunate interaction between how we implement our
messageport "receive loop" and a pattern found in `npm:piscina` (which
angular uses). The gist of it is that piscina uses an atomic wait loop
along with `receiveMessageOnPort` in its worker threads, and as the
worker is getting started, the following incredibly convoluted series of
events occurs:
   1. Parent sends a MessagePort `p` to worker
   2. Parent sends a message `m` to the port `p`
3. Parent notifies the worker with `Atomics.notify` that a new message
is available
   4. Worker receives message, adds "message" listener to port `p`
   5. Adding the listener triggers `MessagePort.start()` on `p`
6. Receive loop in MessagePort.start receives the message `m`, but then
hits an await point and yields (before dispatching the "message" event)
7. Worker continues execution, starts the atomic wait loop, and
immediately receives the existing notification from the parent that a
message is available
8. Worker attempts to receive the new message `m` with
`receiveMessageOnPort`, but this returns `undefined` because the receive
loop already took the message in 6
9. Atomic wait loop continues to next iteration, waiting for the next
message with `Atomic.wait`
10. `Atomic.wait` blocks the worker thread, which prevents the receive
loop from continuing and dispatching the "message" event for the
received message
11. The parent waits for the worker to respond to the first message, and
waits
12. The thread can't make any more progress, and the whole process hangs

The fix I've chosen here (which I don't particularly love, but it works)
is to just delay the `MessagePort.start` call until the end of the event
loop turn, so that the atomic wait loop receives the message first. This
prevents the hang.

---

Those were the main issues causing the hang. There ended up being a few
other small bugs as well, namely `exit` being emitted multiple times,
and not patching up the message port when it's received by
`receiveMessageOnPort`.
2024-10-04 09:26:32 -07:00
..
benches perf(cli): use new deno_core timers (#22569) 2024-03-01 11:15:18 -07:00
00_infra.js refactor(ext): throw new error instead of throw error (#25272) 2024-08-28 22:40:37 +02:00
01_dom_exception.js fix(runtime): use more null proto objects again (#25040) 2024-09-06 12:52:59 +02:00
01_mimesniff.js refactor(ext/web): use relative specifiers (#23024) 2024-03-22 17:21:05 +01:00
02_event.js fix(ext/web): don't ignore capture in EventTarget.removeEventListener (#25788) 2024-09-23 11:19:59 +02:00
02_structured_clone.js fix: Float16Array support (#23512) 2024-04-23 22:54:19 +02:00
02_timers.js fix: reland async context (#25140) 2024-08-29 02:25:38 +00:00
03_abort_signal.js chore: update WPT (#25250) 2024-09-03 23:15:34 -07:00
04_global_interfaces.js refactor(ext/web): use relative specifiers (#23024) 2024-03-22 17:21:05 +01:00
05_base64.js refactor(ext/web): use relative specifiers (#23024) 2024-03-22 17:21:05 +01:00
06_streams.js fix(runtime): don't error child.output() on consumed stream (#25657) 2024-09-16 14:23:40 +02:00
06_streams_types.d.ts chore: add code generation for @types/deno (#25545) 2024-09-23 19:18:52 +00:00
08_text_encoding.js fix(ext/web): make TextDecoderResource use cppgc (#24888) 2024-08-06 07:40:17 +00:00
09_file.js feat(ext/web): add Blob.prototype.bytes() (#24148) 2024-07-02 16:04:08 +02:00
10_filereader.js fix(runtime): use more null proto objects again (#25040) 2024-09-06 12:52:59 +02:00
12_location.js fix(runtime): use more null proto objects again (#25040) 2024-09-06 12:52:59 +02:00
13_message_port.js fix(node): fix worker_threads issues blocking Angular support (#26024) 2024-10-04 09:26:32 -07:00
14_compression.js refactor(ext/web): use relative specifiers (#23024) 2024-03-22 17:21:05 +01:00
15_performance.js perf(ext/web): optimize performance.measure() (#25774) 2024-09-20 16:24:59 -07:00
16_image_data.js refactor(ext/web): use relative specifiers (#23024) 2024-03-22 17:21:05 +01:00
blob.rs chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
Cargo.toml chore: release deno_* crates (#25987) 2024-10-02 14:27:34 +00:00
compression.rs fix(ext/web): make CompressionResource garbage collectable (#24884) 2024-08-06 12:31:11 +05:30
internal.d.ts fix(ext/node) implement receiveMessageOnPort for node:worker_threads (#22766) 2024-03-11 00:23:06 +01:00
lib.deno_web.d.ts chore: add code generation for @types/deno (#25545) 2024-09-23 19:18:52 +00:00
lib.rs fix(ext/web): make TextDecoderResource use cppgc (#24888) 2024-08-06 07:40:17 +00:00
message_port.rs fix(node): fix worker_threads issues blocking Angular support (#26024) 2024-10-04 09:26:32 -07:00
README.md fix(ext/web): use primordials of ES2024 ArrayBuffer transfer (#24396) 2024-07-02 22:27:01 +02:00
stream_resource.rs fix(ext/web): fix potential leak of unread buffers (#23923) 2024-05-21 17:45:33 +00:00
timers.rs BREAKING(permissions): remove --allow-hrtime (#25367) 2024-09-03 11:24:25 +02:00

deno web

Implements timers, as well as the following APIs:

Note: Testing for text encoding is done via WPT in cli/.

Usage Example

From javascript, include the extension's source:

import * as infra from "ext:deno_web/00_infra.js";
import * as DOMException from "ext:deno_web/01_dom_exception.js";
import * as mimesniff from "ext:deno_web/01_mimesniff.js";
import * as event from "ext:deno_web/02_event.js";
import * as structuredClone from "ext:deno_web/02_structured_clone.js";
import * as timers from "ext:deno_web/02_timers.js";
import * as abortSignal from "ext:deno_web/03_abort_signal.js";
import * as globalInterfaces from "ext:deno_web/04_global_interfaces.js";
import * as base64 from "ext:deno_web/05_base64.js";
import * as streams from "ext:deno_web/06_streams.js";
import * as encoding from "ext:deno_web/08_text_encoding.js";
import * as file from "ext:deno_web/09_file.js";
import * as fileReader from "ext:deno_web/10_filereader.js";
import * as location from "ext:deno_web/12_location.js";
import * as messagePort from "ext:deno_web/13_message_port.js";
import * as compression from "ext:deno_web/14_compression.js";
import * as performance from "ext:deno_web/15_performance.js";
import * as imageData from "ext:deno_web/16_image_data.js";

Then assign the properties below to the global scope like this example:

Object.defineProperty(globalThis, "AbortController", {
  value: abortSignal.AbortController,
  enumerable: false,
  configurable: true,
  writable: true,
});
Name Value enumerable configurable writeable
AbortController abortSignal.AbortController false true true
AbortSignal abortSignal.AbortSignal false true true
Blob file.Blob false true true
ByteLengthQueuingStrategy streams.ByteLengthQueuingStrategy
CloseEvent event.CloseEvent false true true
CompressionStream compression.CompressionStream false true true
CountQueuingStrategy streams.CountQueuingStrategy
CustomEvent event.CustomEvent false true true
DecompressionStream compression.DecompressionStream false true true
DOMException DOMException false true true
ErrorEvent event.ErrorEvent false true true
Event event.Event false true true
EventTarget event.EventTarget false true true
File file.File false true true
FileReader fileReader.FileReader false true true
MessageEvent event.MessageEvent false true true
Performance performance.Performance false true true
PerformanceEntry performance.PerformanceEntry false true true
PerformanceMark performance.PerformanceMark false true true
PerformanceMeasure performance.PerformanceMeasure false true true
PromiseRejectionEvent event.PromiseRejectionEvent false true true
ProgressEvent event.ProgressEvent false true true
ReadableStream streams.ReadableStream false true true
ReadableStreamDefaultReader streams.ReadableStreamDefaultReader
TextDecoder encoding.TextDecoder false true true
TextEncoder encoding.TextEncoder false true true
TextDecoderStream encoding.TextDecoderStream false true true
TextEncoderStream encoding.TextEncoderStream false true true
TransformStream streams.TransformStream false true true
MessageChannel messagePort.MessageChannel false true true
MessagePort messagePort.MessagePort false true true
WritableStream streams.WritableStream false true true
WritableStreamDefaultWriter streams.WritableStreamDefaultWriter
WritableStreamDefaultController streams.WritableStreamDefaultController
ReadableByteStreamController streams.ReadableByteStreamController
ReadableStreamBYOBReader streams.ReadableStreamBYOBReader
ReadableStreamBYOBRequest streams.ReadableStreamBYOBRequest
ReadableStreamDefaultController streams.ReadableStreamDefaultController
TransformStreamDefaultController streams.TransformStreamDefaultController
ImageData imageData.ImageData false true true
atob base64.atob true true true
btoa base64.btoa true true true
clearInterval timers.clearInterval true true true
clearTimeout timers.clearTimeout true true true
performance performance.performance true true true
reportError event.reportError true true true
setInterval timers.setInterval true true true
setTimeout timers.setTimeout true true true
structuredClone messagePort.structuredClone true true true

Then from rust, provide: deno_web::deno_web::init_ops_and_esm::<Permissions>(Arc<BlobStore>, Option<Url>) in the extensions field of your RuntimeOptions

Where:

  • Permissions is a struct implementing deno_web::TimersPermission
  • Arc<BlobStore> can be provided by Default::default()
  • Option<Url> provides an optional base URL for certain ops

Dependencies

  • deno_webidl: Provided by the deno_webidl crate
  • deno_console: Provided by the deno_console crate
  • deno_url: Provided by the deno_url crate

Provided ops

Following ops are provided, which can be accessed through Deno.ops:

  • op_base64_decode
  • op_base64_encode
  • op_base64_atob
  • op_base64_btoa
  • op_encoding_normalize_label
  • op_encoding_decode_single
  • op_encoding_decode_utf8
  • op_encoding_new_decoder
  • op_encoding_decode
  • op_encoding_encode_into
  • op_blob_create_part
  • op_blob_slice_part
  • op_blob_read_part
  • op_blob_remove_part
  • op_blob_create_object_url
  • op_blob_revoke_object_url
  • op_blob_from_object_url
  • op_message_port_create_entangled
  • op_message_port_post_message
  • op_message_port_recv_message
  • op_message_port_recv_message_sync
  • op_compression_new
  • op_compression_write
  • op_compression_finish
  • op_now
  • op_defer
  • op_readable_stream_resource_allocate
  • op_readable_stream_resource_allocate_sized
  • op_readable_stream_resource_get_sink
  • op_readable_stream_resource_write_error
  • op_readable_stream_resource_write_buf
  • op_readable_stream_resource_write_sync
  • op_readable_stream_resource_close
  • op_readable_stream_resource_await_close