foster/denoland-deno
1
0
Fork 0
mirror of https://github.com/denoland/deno.git synced 2025-01-08 15:19:40 -05:00
Code Issues Packages 1 Wiki Activity

docs: add npm-Node.js chapter (#11419)

Browse source 
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This commit is contained in:
Kitson Kelly 2021-07-18 16:30:17 +10:00 committed by GitHub
parent e0e26b4101
commit 44084cd0f9
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
9 changed files with 537 additions and 16 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

59
docs/npm_nodejs.md Normal file
View file

@ -0,0 +1,59 @@
# Using npm/Node.js code
While Deno is pretty powerful itself, there is a large eco-system of code in the
[npm](https://npmjs.com/) registry, and many people will want to re-leverage
tools, code and libraries that are built for [Node.js](https://nodejs.org/). In
this chapter we will explore how to use it.
The good news, is that in many cases, it _just works_.
There are some foundational things to understand about differences between
Node.js and Deno that can help in understanding what challenges might be faced:
- Current Node.js supports both CommonJS and ES Modules, while Deno only
supports ES Modules. The addition of stabilized ES Modules in Node.js is
relatively recent and most code written for Node.js is in the CommonJS format.
- Node.js has quite a few built-in modules that can be imported and they are a
fairly expansive set of APIs. On the other hand, Deno focuses on implementing
web standards and where functionality goes beyond the browser, we locate APIs
in a single global `Deno` variable/namespace. Lots of code written for Node.js
expects/depends upon these built-in APIs to be available.
- Node.js has a non-standards based module resolution algorithm, where you can
import bare-specifiers (e.g. `react` or `lodash`) and Node.js will look in
your local and global `node_modules` for a path, introspect the `package.json`
and try to see if there is a module named the right way. Deno resolves modules
the same way a browser does. For local files, Deno expects a full module name,
including the extension. When dealing with remote imports, Deno expects the
web server to do any "resolving" and provide back the media type of the code
(see the
[Determining the type of file](../typescript/overview.md#determining-the-type-of-file)
for more information).
- Node.js effectively doesn't work without a `package.json` file. Deno doesn't
require an external meta-data file to function or resolve modules.
- Node.js doesn't support remote HTTP imports. It expects all 3rd party code to
be installed locally on your machine using a package manager like `npm` into
the local or global `node_modules` folder. Deno supports remote HTTP imports
(as well as `data` and `blob` URLs) and will go ahead and fetch the remote
code and cache it locally, similar to the way a browser works.
In order to help mitigate these differences, we will further explore in this
chapter:
- Using the [`std/node`](./npm_nodejs/std_node.md) modules of the Deno standard
library to "polyfill" the built-in modules of Node.js
- Using [CDNs](./npm_nodejs/cdns.md) to access the vast majority of npm packages
in ways that work under Deno.
- How [import maps](./npm_nodejs/import_maps.md) can be used to provide "bare
specifier" imports like Node.js under Deno, without needing to use a package
manager to install packages locally.
- And finally, a general section of [frequently asked
questions][./npm_nodejs/faqs.md]
That being said, there are some differences that cannot be overcome:
- Node.js has a plugin system that is incompatible with Deno, and Deno will
never support Node.js plugins. If the Node.js code you want to use requires a
"native" Node.js plugin, it won't work under Deno.
- Node.js has some built in modules (e.g. like `vm`) that are effectively
incompatible with the scope of Deno and therefore there aren't easy ways to
provide a _polyfill_ of the functionality in Deno.

186
docs/npm_nodejs/cdns.md Normal file
View file

@ -0,0 +1,186 @@
## Packages from CDNs
Because Deno supports remote HTTP modules, and content delivery networks (CDNs)
can be powerful tools to transform code, the combination allows an easy way to
access code in the npm registry via Deno, usually in a way that works with Deno
without any further actions, and often enriched with TypeScript types. In this
section we will explore that in detail.
### What about `deno.land/x/`?
The [`deno.land/x/`](https://deno.land/x/) is a public registry for code,
hopefully code written specifically for Deno. It is a public registry though and
all it does is "redirect" Deno to the location where the code exists. It doesn't
transform the code in any way. There is a lot of great code on the registry, but
at the same time, there is some code that just isn't well maintained (or doesn't
work at all). If you are familiar with the npm registry, you know that as well,
there are varying degrees of quality.
Because it simply serves up the original published source code, it doesn't
really help when trying to use code that didn't specifically consider Deno when
authored.
### Deno "friendly" CDNs
Deno friendly content delivery networks (CDNs) not only host packages from npm,
they provide them in a way that maximizes their integration to Deno. They
directly address some of the challenges in consuming code written for Node.js:
- The provide packages and modules in the ES Module format, irrespective of how
they are published on npm.
- They resolve all the dependencies as the modules are served, meaning that all
the Node.js specific module resolution logic is handled by the CDN.
- Often, they inform Deno of type definitions for a package, meaning that Deno
can use them to type check your code and provide a better development
experience.
- The CDNs also "polyfill" the built-in Node.js modules, making a lot of code
that leverages the built-in Node.js modules _just work_.
- The CDNs deal with all the semver matching for packages that a package manager
like `npm` would be required for a Node.js application, meaning you as a
developer can express your 3rd party dependency versioning as part of the URL
you use to import the package.
#### esm.sh
[esm.sh](https://esm.sh/) is a CDN that was specifically designed for Deno,
though addressing the concerns for Deno also makes it a general purpose CDN for
accessing npm packages as ES Module bundles. esm.sh uses
[esbuild](https://esbuild.github.io/) to take an arbitrary npm package and
ensure that it is consumable as an ES Module. In many cases you can just import
the npm package into your Deno application:
```ts
import React from "https://esm.sh/react";
export default class A extends React.Component {
render() {
return (
<div></div>
);
}
}
```
esm.sh supports the use of both specific versions of packages, as well as
[semver](https://semver.npmjs.com/) versions of packages, so you can express
your dependency in a similar way you would in a `package.json` file when you
import it. For example, to get a specific version of a package:
```ts
import React from "https://esm.sh/react@17.0.2";
```
Or to get the latest patch release of a minor release:
```ts
import React from "https://esm.sh/react@~16.13.0";
```
esm.sh uses the `std/node` polyfills to replace the built-in modules in Node.js,
meaning that code that uses those built-in modules will have the same
limitations and caveats as those modules in `std/node`.
esm.sh also automatically sets a header which Deno recognizes that allows Deno
to be able to retrieve type definitions for the package/module. See
[Using `X-TypeScript-Types` header](../typescript/types.md#using-x-typescript-types-header)
in this manual for more details on how this works.
The CDN is also a good choice for people who develop in mainland China, as the
hosting of the CDN is specifically designed to work with "the great firewall of
China", as well as esm.sh provides information on self hosting the CDN as well.
Check out the [esm.sh homepage](https://esm.sh/) for more detailed information
on how the CDN can be used and what features it has.
#### Skypack
[Skypack.dev](https://www.skypack.dev/) is designed to make development overall
easier by not requiring packages to be installed locally, even for Node.js
development, and to make it easy to create web and Deno applications that
leverage code from the npm registry.
Skypack has a great way of discovering packages in the npm registry, by
providing a lot of contextual information about the package, as well as a
"scoring" system to try to help determine if the package follows best-practices.
Skypack detects Deno's user agent when requests for modules are received and
ensures the code served up is tailored to meet the needs of Deno. The easiest
way to load a package is to use the
[lookup URL](https://docs.skypack.dev/skypack-cdn/api-reference/lookup-urls) for
the package:
```ts
import React from "https://cdn.skypack.dev/react";
export default class A extends React.Component {
render() {
return (
<div></div>
);
}
}
```
Lookup URLs can also contain the [semver](https://semver.npmjs.com/) version in
the URL:
```ts
import React from "https://cdn.skypack.dev/react@~16.13.0";
```
By default, Skypack does not set the types header on packages. In order to have
the types header set, which is automatically recognized by Deno, you have to
append `?dts` to the URL for that package:
```ts
import { pathToRegexp } from "https://cdn.skypack.dev/path-to-regexp?dts";
const re = pathToRegexp("/path/:id");
```
See
[Using `X-TypeScript-Types` header](../typescript/types.md#using-x-typescript-types-header)
in this manual for more details on how this works.
Skypack docs have a
[specific page on usage with Deno](https://docs.skypack.dev/skypack-cdn/code/deno)
for more information.
### Other CDNs
There are a couple of other CDNs worth mentioning.
#### UNPKG
[UNPKG](https://unpkg.com/) is the most well known CDN for npm packages. For
packages that include an ES Module distribution for things like the browsers,
many of them can be used directly off of UNPKG. That being said, everything
available on UNPKG is available on more Deno friendly CDNs.
#### JSPM
The [jspm.io](https://jspm.io) CDN is specifically designed to provide npm and
other registry packages as ES Modules in a way that works well with import maps.
While it doesn't currently cater to Deno, the fact that Deno can utilize import
maps, allows you to use the [JSPM.io generator](https://generator.jspm.io/) to
generate an import-map of all the packages you want to use and have them served
up from the CDN.
### Considerations
While CDNs can make it easy to allow Deno to consume packages and modules from
the npm registry, there can still be some things to consider:
- Deno does not (and will not) support Node.js plugins. If the package requires
a native plugin, it won't work under Deno.
- Dependency management can always be a bit of a challenge and a CDN can make it
a bit more obfuscated what dependencies are there. You can always use
`deno info` with the module or URL to get a full breakdown of how Deno
resolves all the code.
- While the Deno friendly CDNs try their best to serve up types with the code
for consumption with Deno, lots of types for packages conflict with other
packages and/or don't consider Deno, which means you can often get strange
diagnostic message when type checking code imported from these CDNs, though
skipping type checking will result in the code working perfectly fine. This is
a fairly complex topic and is covered in the
[Types and type declarations](../typescript/types.md) section of the manual.

55
docs/npm_nodejs/faqs.md Normal file
View file

@ -0,0 +1,55 @@
## Frequently asked questions
### Getting errors when type checking like `cannot find namespace NodeJS`
One of the modules you are using has type definitions that depend upon the
NodeJS global namespace, but those types don't include the NodeJS global
namespace in their types.
The quickest fix is to skip type checking. You can do this by using the
`--no-check` flag.
Skipping type checking might not be acceptable though. You could try to load the
Node.js types yourself. For example from UNPKG it would look something like
this:
```ts
import type {} from "https://unpkg.com/@types/node/index.d.ts";
```
Or from esm.sh:
```ts
import type {} from "https://esm.sh/@types/node/index.d.ts";
```
Or from Skypack:
```ts
import type {} from "https://cdn.skypack.dev/@types/node/index.d.ts";
```
You could also try to provide only specifically what the 3rd party package is
missing. For example the package `@aws-sdk/client-dynamodb` has a dependency on
the `NodeJS.ProcessEnv` type in its type definitions. In one of the modules of
your project that imports it as a dependency, you could put something like this
in there which will solve the problem:
```ts
declare global {
namespace NodeJS {
type ProcessEnv = Record<string, string>;
}
}
```
### Getting type errors like cannot find `document` or `HTMLElement`
The library you are using has dependencies on the DOM. This is common for
packages that are designed to run in a browser as well as server-side. By
default, Deno only includes the libraries that are directly supported. Assuming
the package properly identifies what environment it is running in at runtime it
is "safe" to use the DOM libraries to type check the code. For more information
on this, check out the
[Targeting Deno and the Browser](../typescript/configuration.md#targeting-deno-and-the-browser)
section of the manual.

112
docs/npm_nodejs/import_maps.md Normal file
View file

@ -0,0 +1,112 @@
## Using import maps
Deno supports [import maps](../linking_to_external_code/import_maps.md) which
allow you to supply Deno with information about how to resolve modules that
overrides the default behavior. Import maps are a web platform standard that is
increasingly being included natively in browsers. They are specifically useful
with adapting Node.js code to work well with Deno, as you can use import maps to
map "bare" specifiers to a specific module.
When coupled with Deno friendly [CDNs](./cdns.md) import maps can be a powerful
tool in managing code and dependencies without need of a package management
tool.
### Bare and extension-less specifiers
Deno will only load a fully qualified module, including the extension. The
import specifier needs to either be relative or absolute. Specifiers that are
neither relative or absolute are often called "bare" specifiers. For example
`"./lodash/index.js"` is a relative specifier and
`https://cdn.skypack.dev/lodash` is an absolute specifier. Where is `"lodash"`
would be a bare specifier.
Also Deno requires that for local modules, the module to load is fully
resolve-able. When an extension is not present, Deno would have to "guess" what
the author intended to be loaded. For example does `"./lodash"` mean
`./lodash.js`, `./lodash.ts`, `./lodash.tsx`, `./lodash.jsx`,
`./lodash/index.js`, `./lodash/index.ts`, `./lodash/index.jsx`, or
`./lodash/index.tsx`?
When dealing with remote modules, Deno allows the CDN/web server define whatever
semantics around resolution the server wants to define. It just treats a URL,
including its query string, as a "unique" module that can be loaded. It expects
the CDN/web server to provide it with a valid media/content type to instruct
Deno how to handle the file. More information on how media types impact how Deno
handles modules can be found in the
[Determining the type of file](../typescript/overview.md#determining-the-type-of-file)
section of the manual.
Node.js does have defined semantics for resolving specifiers, but they are
complex, assume unfettered access to the local file system to query it. Deno has
chosen not to go down that path.
But, import maps can be used to provide some of the ease of the developer
experience if you wish to use bare specifiers. For example, if we want to do the
following in our code:
```ts
import lodash from "lodash";
```
We can accomplish this using an import map, and we don't even have to install
the `lodash` package locally. We would want to create a JSON file (for example
**import_map.json**) with the following:
```json
{
"imports": {
"lodash": "https://cdn.skypack.dev/lodash"
}
}
```
And we would run our program like:
```
> deno run --import-map ./import_map.json example.ts
```
If you wanted to manage the versions in the import map, you could do this as
well. For example if you were using Skypack CDN, you can used a
[pinned URL](https://docs.skypack.dev/skypack-cdn/api-reference/pinned-urls-optimized)
for the dependency in your import map. For example, to pin to lodash version
4.17.21 (and minified production ready version), you would do this:
```json
{
"imports": {
"lodash": "https://cdn.skypack.dev/pin/lodash@v4.17.21-K6GEbP02mWFnLA45zAmi/mode=imports,min/optimized/lodash.js"
}
}
```
### Overriding imports
The other situation where import maps can be very useful is the situation where
you have tried your best to make something work, but have failed. For example
you are using an npm package which has a dependency on some code that just
doesn't work under Deno, and you want to substitute another module that
"polyfills" the incompatible APIs.
For example, let's say we have a package that is using a version of the built in
`"fs"` module that we have a local module we want to replace it with when it
tries to import it, but we want other code we are loading to use the standard
library replacement module for `"fs"`. We would want to create an import map
that looked something like this:
```ts
{
"imports": {
"fs": "https://deno.land/std@$STD_VERSION/node/fs.ts"
},
"scopes": {
"https://deno.land/x/example": {
"fs": "./patched/fs.ts"
}
}
}
```
Import maps can be very powerful, check out the official
[standards README](https://github.com/WICG/import-maps#the-import-map) for more
information.

100
docs/npm_nodejs/std_node.md Normal file
View file

@ -0,0 +1,100 @@
## The `std/node` library
The `std/node` part of the Deno standard library is a Node.js compatibility
layer. It's primary focus is providing "polyfills" for Node.js's
[built-in modules](https://github.com/denoland/deno_std/tree/main/node#supported-builtins).
It also provides a mechanism for loading CommonJS modules into Deno.
The library is most useful when trying to use your own or private code that was
written for Node.js. If you are trying to consume public npm packages, you are
likely to get a better result using a [CDN](./cdns.md).
### Node.js built-in modules
The standard library provides several "replacement" modules for Node.js built-in
modules. These either replicate the functionality of the built-in or they call
the Deno native APIs, returning an interface that is compatible with Node.js.
The standard library provides modules for the the following built-ins:
- [`assert`](https://doc.deno.land/https/deno.land/std/node/assert.ts)
(_partly_)
- [`buffer`](https://doc.deno.land/https/deno.land/std/node/buffer.ts)
- [`child_process`](https://doc.deno.land/https/deno.land/std/node/child_process.ts)
(_partly_)
- [`console`](https://doc.deno.land/https/deno.land/std/node/console.ts)
(_partly_)
- [`constants`](https://doc.deno.land/https/deno.land/std/node/constants.ts)
(_partly_)
- [`crypto`](https://doc.deno.land/https/deno.land/std/node/crypto.ts)
(_partly_)
- [`events`](https://doc.deno.land/https/deno.land/std/node/events.ts)
- [`fs`](https://doc.deno.land/https/deno.land/std/node/fs.ts) (_partly_)
- [`module`](https://doc.deno.land/https/deno.land/std/node/module.ts)
- [`os`](https://doc.deno.land/https/deno.land/std/node/os.ts) (_partly_)
- [`path`](https://doc.deno.land/https/deno.land/std/node/path.ts)
- [`process`](https://doc.deno.land/https/deno.land/std/node/process.ts)
(_partly_)
- [`querystring`](https://doc.deno.land/https/deno.land/std/node/querystring.ts)
- [`stream`](https://doc.deno.land/https/deno.land/std/node/stream.ts)
- [`string_decoder`](https://doc.deno.land/https/deno.land/std/node/string_decoder.ts)
- [`timers`](https://doc.deno.land/https/deno.land/std/node/timers.ts)
- [`tty`](https://doc.deno.land/https/deno.land/std/node/tty.ts) (_partly_)
- [`url`](https://doc.deno.land/https/deno.land/std/node/url.ts)
- [`util`](https://doc.deno.land/https/deno.land/std/node/util.ts) (_partly_)
In addition, there is the
[`std/node/global.ts`](https://doc.deno.land/https/deno.land/std/node/global.ts)
module which provides some of the Node.js globals like `global`, `process`, and
`Buffer`.
If you want documentation for any of the modules, you can simply type `deno doc`
and the URL of the module in your terminal:
```
> deno doc https://deno.land/std/assert.ts
```
### Loading CommonJS modules
Deno only supports natively loading ES Modules, but a lot of Node.js code is
still written in the CommonJS format. As mentioned above, for public npm
packages, it is often better to use a CDN to transpile CommonJS modules to ES
Modules for consumption by Deno. Not only do you get reliable conversion plus
all the dependency resolution and management without requiring a package manager
to install the packages on your local machine, you get the advantages of being
able to share your code easier without requiring other users to setup some of
the Node.js infrastructure to consume your code with Node.js.
That being said, the built-in Node.js module `"module"` provides a function
named `createRequire()` which allows you to create a Node.js compatible
`require()` function which can be used to load CommonJS modules, as well as use
the same resolution logic that Node.js uses when trying to load a module.
`createRequire()` will also install the Node.js globals for you.
Example usage would look like this:
```ts
import { createRequire } from "https://deno.land/std@$STD_VERSION/node/module.ts";
// import.meta.url will be the location of "this" module (like `__filename` in
// Node.js), and then serve as the root for your "package", where the
// `package.json` is expected to be, and where the `node_modules` will be used
// for resolution of packages.
const require = createRequire(import.meta.url);
// Loads the built-in module Deno "replacement":
const path = require("path");
// Loads a CommonJS module (without specifying the extension):
const cjsModule = require("./my_mod");
// Uses Node.js resolution in `node_modules` to load the package/module. The
// package would need to be installed locally via a package management tool
// like npm:
const leftPad = require("left-pad");
```
When modules are loaded via the created `require()`, they are executed in a
context which is similar to a Node.js context, which means that a lot of code
written targeting Node.js will work.

9
docs/toc.json
View file

@ -37,6 +37,15 @@
"import_maps": "Import maps"
}
},
"npm_nodejs": {
"name": "Using npm/Node.js code",
"children": {
"std_node": "The std/node library",
"cdns": "Packages from CDNs",
"import_maps": "Using import maps",
"faqs": "Frequently asked questions"
}
},
"typescript": {
"name": "Using TypeScript",
"children": {

8
docs/typescript/faqs.md
View file

@ -18,10 +18,10 @@ If you are using `tsc` as stand-alone, the setting to use is `"isolatedModules"`
and setting it to `true` to help ensure that your code can be properly handled
by Deno.
One of the ways to deal with the extension and the lack of _magical_ resolution
is to use [import maps](../linking_to_external_code/import_maps.md) which would
allow you to specify "packages" of bare specifiers which then Deno could resolve
and load.
One of the ways to deal with the extension and the lack of Node.js non-standard
resolution logic is to use
[import maps](../linking_to_external_code/import_maps.md) which would allow you
to specify "packages" of bare specifiers which then Deno could resolve and load.
### What version(s) of TypeScript does Deno support?

10
docs/typescript/overview.md
View file

@ -152,8 +152,8 @@ bypass type checking all together.
### Type resolution
One of the core design principles of Deno is to avoid "magical" resolution, and
this applies to type resolution as well. If you want to utilise JavaScript that
has type definitions (e.g. a `.d.ts` file), you have to explicitly tell Deno
about this. The details of how this is accomplished are covered in the
[Types and type declarations](./types.md) section.
One of the core design principles of Deno is to avoid non-standard module
resolution, and this applies to type resolution as well. If you want to utilise
JavaScript that has type definitions (e.g. a `.d.ts` file), you have to
explicitly tell Deno about this. The details of how this is accomplished are
covered in the [Types and type declarations](./types.md) section.

14
docs/typescript/types.md
View file

@ -1,12 +1,12 @@
## Types and Type Declarations
One of the design principles of Deno is no _magical_ resolution. When TypeScript
is type checking a file, it only cares about the types for the file, and the
`tsc` compiler has a lot of logic to try to resolve those types. By default, it
expects _ambiguous_ module specifiers with an extension, and will attempt to
look for the file under the `.ts` specifier, then `.d.ts`, and finally `.js`
(plus a whole other set of logic when the module resolution is set to `"node"`).
Deno deals with explicit specifiers.
One of the design principles of Deno is no non-standard module resolution. When
TypeScript is type checking a file, it only cares about the types for the file,
and the `tsc` compiler has a lot of logic to try to resolve those types. By
default, it expects _ambiguous_ module specifiers with an extension, and will
attempt to look for the file under the `.ts` specifier, then `.d.ts`, and
finally `.js` (plus a whole other set of logic when the module resolution is set
to `"node"`). Deno deals with explicit specifiers.
This can cause a couple problems though. For example, let's say I want to
consume a TypeScript file that has already been transpiled to JavaScript along