1
0
Fork 0
mirror of https://github.com/denoland/deno.git synced 2024-12-23 07:44:48 -05:00
denoland-deno/ext/node/polyfills
Divy Srivastava 1ba88a7892
perf(ext/node): improve Buffer from string performance (#24567)
Fixes https://github.com/denoland/deno/issues/24323

- Use a Buffer pool for `fromString`
- Implement fast call base64 writes
- Direct from string `create` method for each encoding op

```
$ deno bench -A bench.mjs # 1.45.1+fee4d3a
cpu: Apple M1 Pro
runtime: deno 1.45.1+fee4d3a (aarch64-apple-darwin)

benchmark                time (avg)             (min … max)       p75       p99      p999
----------------------------------------------------------- -----------------------------
Buffer.from base64      550 ns/iter     (490 ns … 1'265 ns)    572 ns    606 ns  1'265 ns
Buffer#write base64     285 ns/iter       (259 ns … 371 ns)    307 ns    347 ns    360 ns

$ ~/gh/deno/target/release/deno bench -A bench.mjs # this PR
cpu: Apple M1 Pro
runtime: deno dev (aarch64-apple-darwin)

benchmark                time (avg)             (min … max)       p75       p99      p999
----------------------------------------------------------- -----------------------------
Buffer.from base64      151 ns/iter       (145 ns … 770 ns)    148 ns    184 ns    648 ns
Buffer#write base64   62.58 ns/iter     (60.79 ns … 157 ns)  61.65 ns  75.79 ns    141 ns

$ node bench.mjs # v22.4.0
cpu: Apple M1 Pro
runtime: node v22.4.0 (arm64-darwin)

benchmark                time (avg)             (min … max)       p75       p99      p999
----------------------------------------------------------- -----------------------------
Buffer.from base64      163 ns/iter     (96.92 ns … 375 ns)  99.45 ns    127 ns    220 ns
Buffer#write base64   75.48 ns/iter     (74.97 ns … 134 ns)  75.17 ns  81.83 ns  96.84 ns
```
2024-07-30 18:09:55 +05:30
..
_fs perf(ext/node): optimize fs.exists[Sync] (#24613) 2024-07-17 18:05:51 +05:30
_process fix(ext/node): use primordials in ext/node/polyfills/_process/ (#24282) 2024-06-20 17:26:34 +10:00
_util fix(node): util.callbackify (#22200) 2024-02-01 08:51:10 +05:30
assert chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
dns chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
fs fix(node): add cp to fs/promises (#22263) 2024-02-06 12:32:58 +05:30
internal perf(ext/node): improve Buffer from string performance (#24567) 2024-07-30 18:09:55 +05:30
internal_binding perf(ext/node): improve Buffer from string performance (#24567) 2024-07-30 18:09:55 +05:30
path chore: sync up Node.js test files for v20.11.1 (#24066) 2024-06-11 11:41:44 +00:00
readline chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
stream chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
timers chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
util chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
00_globals.js chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
01_require.js fix: actually add missing node:readline/promises module (#24772) 2024-07-28 15:41:10 -04:00
02_init.js fix(ext/node): don't rely on Deno.env to read NODE_DEBUG (#23694) 2024-05-05 16:16:02 +02:00
_brotli.js perf(ext/node): improve Buffer from string performance (#24567) 2024-07-30 18:09:55 +05:30
_events.d.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
_events.mjs fix: missing emitWarning import (#24587) 2024-07-19 23:35:07 +00:00
_global.d.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
_http_agent.mjs refactor: rename built-in node modules from ext:deno_node/ to node: (#19680) 2023-07-02 20:19:30 +02:00
_http_common.ts fix(ext/node): use primordials in ext/node/polyfills/_http_common.ts (#24281) 2024-06-20 17:26:14 +10:00
_http_outgoing.ts perf(ext/node): improve Buffer from string performance (#24567) 2024-07-30 18:09:55 +05:30
_next_tick.ts refactor: use core.ensureFastOps() (#21888) 2024-01-10 15:37:25 -07:00
_readline.d.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
_readline.mjs refactor: rename built-in node modules from ext:deno_node/ to node: (#19680) 2023-07-02 20:19:30 +02:00
_readline_shared_types.d.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
_stream.d.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
_stream.mjs chore: sync up Node.js test files for v20.11.1 (#24066) 2024-06-11 11:41:44 +00:00
_tls_common.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
_tls_wrap.ts fix(ext/node): Correctly send ALPN on node TLS connections (#23434) 2024-04-18 09:37:47 -06:00
_utils.ts fix(ext/node): use primordials in ext/node/polyfills/_utils.ts (#24253) 2024-06-19 18:29:26 +10:00
_zlib.mjs fix(node/zlib): accept dataview and buffer in zlib bindings (#21756) 2024-01-03 19:03:51 +05:30
_zlib_binding.mjs fix(ext/node): use cppgc for node:zlib (#24267) 2024-06-20 10:31:53 +05:30
assert.ts fix(node/assert): throws not checking error instance (#24466) 2024-07-08 21:28:39 +02:00
assertion_error.ts feat: Deno.{stdin,stdout,stderr}.isTerminal(), deprecate Deno.isatty() (#22011) 2024-01-24 00:01:56 +01:00
async_hooks.ts fix(node): stub AsyncResource.emitDestroy() (#23802) 2024-05-15 00:42:43 +02:00
buffer.ts feat(node): buffer isUtf8/isAscii (#23928) 2024-05-21 15:50:59 -07:00
child_process.ts fix(ext/node): strip --enable-source-maps from argv (#22743) 2024-03-06 13:40:47 +00:00
cluster.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
console.ts refactor: split runtime/98_global_scope.js (#21785) 2024-01-06 14:40:26 +01:00
constants.ts fix(ext/node): add crypto and zlib constants (#24151) 2024-06-09 02:02:47 +02:00
crypto.ts fix(ext/node): add crypto.subtle (#23027) 2024-03-22 18:56:36 +05:30
dgram.ts fix(ext/node/net): emit error before close when connection is refused (#24656) 2024-07-24 20:33:45 +09:00
diagnostics_channel.js fix(ext/node): better support for node:diagnostics_channel module (#24088) 2024-06-15 00:07:02 +02:00
dns.ts fix(ext/node): Match punycode module behavior to node (#22847) 2024-03-11 15:49:43 -07:00
domain.ts fix(ext/node): polyfill node:domain module (#23088) 2024-04-03 21:37:10 +02:00
events.ts fix(ext/node): implement EventEmitterAsyncResource (#22994) 2024-03-20 11:20:18 +05:30
fs.ts fix(node): Implement fs.lchown (and process.getegid) (#24418) 2024-07-05 18:32:51 +00:00
http.ts refactor(ext/node): create separate ops for node:http module (#24788) 2024-07-30 14:34:52 +02:00
http2.ts fix(ext/node): send data frame with end_stream flag on _final call (#24147) 2024-06-10 16:30:56 +02:00
https.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
inspector.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
net.ts feat(ext/node): add BlockList & SocketAddress classes (#24229) 2024-06-18 10:46:13 +00:00
os.ts fix(node): Run node compat tests listed in the ignore field (and fix the ones that fail) (#24631) 2024-07-24 03:12:08 +00:00
path.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
perf_hooks.ts fix(node/perf_hooks): stub eventLoopUtilization (#24501) 2024-07-10 19:47:45 +02:00
process.ts fix(node): Run node compat tests listed in the ignore field (and fix the ones that fail) (#24631) 2024-07-24 03:12:08 +00:00
punycode.ts fix(ext/node): Match punycode module behavior to node (#22847) 2024-03-11 15:49:43 -07:00
querystring.js fix(ext/node): querystring stringify without encode callback (#21740) 2024-01-02 04:54:11 +00:00
readline.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
README.md chore: move tools/node_compat to tests/node_compat/runner (#23025) 2024-04-03 09:24:55 +11:00
repl.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
stream.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
string_decoder.ts perf(ext/node): improve Buffer from string performance (#24567) 2024-07-30 18:09:55 +05:30
sys.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
testing.ts fix(ext/node): use primordials in ext/node/polyfills/testing.ts (#24310) 2024-06-24 15:49:32 +10:00
timers.ts fix(ext/node): use primordials in ext/node/polyfills/timers.ts (#24311) 2024-06-24 15:50:02 +10:00
tls.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
tty.js fix(node): support tty.hasColors() and tty.getColorDepth() (#24619) 2024-07-19 12:39:05 +02:00
url.ts chore: sync up Node.js test files for v20.11.1 (#24066) 2024-06-11 11:41:44 +00:00
util.ts fix(node): util.callbackify (#22200) 2024-02-01 08:51:10 +05:30
v8.ts fix(node/v8): stub serializer methods (#24502) 2024-07-10 19:46:40 +02:00
vm.ts fix(ext/node): node:vm contexts (#23202) 2024-04-09 21:24:25 +05:30
wasi.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00
worker_threads.ts fix(node/worker_threads): support port.once() (#24725) 2024-07-25 23:06:19 +02:00
zlib.ts chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00

Deno Node.js compatibility

This module is meant to have a compatibility layer for the Node.js standard library.

Warning: Any function of this module should not be referred anywhere in the Deno standard library as it's a compatibility module.

Supported modules

  • assert
  • assert/strict partly
  • async_hooks partly
  • buffer
  • child_process partly
  • cluster partly
  • console partly
  • constants partly
  • crypto partly
  • dgram partly
  • diagnostics_channel partly
  • dns partly
  • events
  • fs partly
  • fs/promises partly
  • http partly
  • http2
  • https partly
  • inspector partly
  • module
  • net
  • os partly
  • path
  • path/posix
  • path/win32
  • perf_hooks
  • process partly
  • punycode
  • querystring
  • readline
  • repl partly
  • stream
  • stream/promises
  • stream/web partly
  • string_decoder
  • sys
  • timers
  • timers/promises
  • tls
  • trace_events
  • tty partly
  • url
  • util partly
  • util/types partly
  • v8
  • vm partly
  • wasi
  • webcrypto
  • worker_threads
  • zlib
  • node globals partly

Deprecated

These modules are deprecated in Node.js and will probably not be polyfilled:

  • domain
  • freelist

Experimental

These modules are experimental in Node.js and will not be polyfilled until they are stable:

  • diagnostics_channel
  • async_hooks
  • policies
  • trace_events
  • wasi
  • webcrypto

CommonJS modules loading

createRequire(...) is provided to create a require function for loading CJS modules. It also sets supported globals.

import { createRequire } from "node:module";

const require = createRequire(import.meta.url);
// Loads native module polyfill.
const path = require("path");
// Loads extensionless module.
const cjsModule = require("./my_mod");
// Visits node_modules.
const leftPad = require("left-pad");

Contributing

Setting up the test runner and running tests

See tests/node_compat/runner/README.md.

Best practices

When converting from promise-based to callback-based APIs, the most obvious way is like this:

promise.then((value) => callback(null, value)).catch(callback);

This has a subtle bug - if the callback throws an error, the catch statement will also catch that error, and the callback will be called twice. The correct way to do it is like this:

promise.then((value) => callback(null, value), callback);

The second parameter of then can also be used to catch errors, but only errors from the existing promise, not the new one created by the callback.

If the Deno equivalent is actually synchronous, there's a similar problem with try/catch statements:

try {
  const value = process();
  callback(null, value);
} catch (err) {
  callback(err);
}

Since the callback is called within the try block, any errors from it will be caught and call the callback again.

The correct way to do it is like this:

let err, value;
try {
  value = process();
} catch (e) {
  err = e;
}
if (err) {
  callback(err); // Make sure arguments.length === 1
} else {
  callback(null, value);
}

It's not as clean, but prevents the callback being called twice.

Remaining Tests

Node compatibility can be measured by how many native Node tests pass. If you'd like to know what you can work on, check out the list of Node tests remaining here.