1
0
Fork 0
mirror of https://github.com/denoland/deno.git synced 2024-12-26 00:59:24 -05:00

Add Docs.md

Remove stuff from Roadmap that has been implemented.
This commit is contained in:
Ryan Dahl 2018-10-18 02:05:33 -04:00
parent 8f9d407e22
commit b5d7e38f9b
6 changed files with 288 additions and 411 deletions

View file

@ -1,7 +1,17 @@
# Thank you...
# Contributing To Deno
Thank you for your interest in contributing to Deno. Here are a few key areas to
consider when contributing to Deno.
Check [the roadmap](https://github.com/denoland/deno/blob/master/Roadmap.md)
before contributing.
Please don't make [the benchmarks](https://denoland.github.io/deno/) worse.
Ask for help in the issues or on the
[chat room](https://gitter.im/denolife/Lobby).
Progress towards future releases is tracked
[here](https://github.com/denoland/deno/milestones).
Docs are [here](https://github.com/denoland/deno/blob/master/Docs.md).
## Submitting a pull request

View file

@ -1,11 +1,3 @@
<!--
Thank you for your pull request. Before submitting, please make sure the following is done.
1. Ensure ./tools/test.py passes.
2. Format your code with ./tools/format.py
3. Make sure ./tools/lint.py passes.
And please read: https://github.com/denoland/deno/blob/master/.github/CONTRIBUTING.md
https://github.com/denoland/deno/blob/master/.github/CONTRIBUTING.md
-->

262
Docs.md Normal file
View file

@ -0,0 +1,262 @@
# Deno Docs
## Install
Deno works on OSX, Linux, and Windows. We provide binary download scripts:
With Python:
```
curl -sSf https://raw.githubusercontent.com/denoland/deno_install/master/install.py | python
```
See also [deno_install](https://github.com/denoland/deno_install).
With PowerShell:
```powershell
iex (iwr https://raw.githubusercontent.com/denoland/deno_install/master/install.ps1)
```
_Note: Depending on your security settings, you may have to run
`Set-ExecutionPolicy RemoteSigned -Scope CurrentUser` first to allow downloaded
scripts to be executed._
Try it:
```
> deno http://deno.land/thumb.ts
```
## API Reference
To get an exact reference of deno's runtime API, run the following in the
command line:
```
deno --types
```
In case you don't have it installed yet, but are curious, here is an out-of-date
copy of the output: https://gist.github.com/78855aeeaddeef7c1fce0aeb8ffef8b2
(We do not yet have an HTML version of this. See
https://github.com/denoland/deno/issues/573)
## Examples
### Example: An implementation of the unix "cat" program
The copy here is actualy zero-copy. That is, it reads data from the socket and
writes it back to it without ever calling a memcpy() or similiar.
```ts
import * as deno from "deno";
for (let i = 1; i < deno.args.length; i++) {
let filename = deno.args[i];
let file = await deno.open(filename);
await deno.copy(deno.stdout, file);
}
```
### Example: A TCP Server echo server
The copy here is actualy zero-copy. That is, it reads data from the socket and
writes it back to it without ever calling a memcpy() or similiar.
```ts
import { listen, accept, copy } from "deno";
const listener = listen("tcp", ":8080");
while (true) {
const conn = await listener.accept();
deno.copy(conn, conn);
}
// TODO top level await doesn't work yet.
```
## How to Profile Deno.
```sh
# Make sure we're only building release.
export DENO_BUILD_MODE=release
# Build deno and V8's d8.
./tools/build.py d8 deno
# Start the program we want to benchmark with --prof
./out/release/deno tests/http_bench.ts --allow-net --prof &
# Exercise it.
third_party/wrk/linux/wrk http://localhost:4500/
kill `pgrep deno`
# When supplying --prof, V8 will write a file in the current directory that
# looks like this isolate-0x7fad98242400-v8.log
# To examine this file
D8_PATH=out/release/ ./third_party/v8/tools/linux-tick-processor
isolate-0x7fad98242400-v8.log
```
## Build instructions (for advanced users only)
### Prerequisists:
To ensure reproducible builds, Deno has most of its dependencies in a git
submodule. However, you need to install separately:
1. [Rust](https://www.rust-lang.org/en-US/install.html)
2. [Node](http://nodejs.org/)
3. Python 2.
[Not 3](https://github.com/denoland/deno/issues/464#issuecomment-411795578).
4. [ccache](https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/ccache)
(Optional but helpful for speeding up rebuilds of V8.).
5. Extra steps for Windows users:
1. Add `python.exe` to `PATH`. E.g. `set PATH=%PATH%;C:\Python27\python.exe`
2. Get [VS Community 2017](https://www.visualstudio.com/downloads/), make
sure to select the option to install C++ tools and the Windows SDK
3. Enable `Debugging Tools for Windows`, Goto Control Panel -> Windows 10 SDK
-> Right-Click -> Change -> Change -> Check Debugging Tools for Windows ->
Change -> Finish
6. Before running `./tools/format.py`, make sure you have `yapf` installed for
the current Python and `rustfmt` for Rust. They can be installed by:
pip install yapf rustup component add rustfmt-preview
### Build:
# Fetch deps.
git clone --recurse-submodules https://github.com/denoland/deno.git
cd deno
./tools/setup.py
# Build.
./tools/build.py
# Run.
./out/debug/deno tests/002_hello.ts
# Test.
./tools/test.py
# Format code.
./tools/format.py
Other useful commands:
# Call ninja manually.
./third_party/depot_tools/ninja -C out/debug
# Build a release binary.
DENO_BUILD_MODE=release ./tools/build.py :deno
# List executable targets.
./third_party/depot_tools/gn ls out/debug //:* --as=output --type=executable
# List build configuation.
./third_party/depot_tools/gn args out/debug/ --list
# Edit build configuration.
./third_party/depot_tools/gn args out/debug/
# Describe a target.
./third_party/depot_tools/gn desc out/debug/ :deno
./third_party/depot_tools/gn help
Env vars: `DENO_BUILD_MODE`, `DENO_BUILD_PATH`, `DENO_BUILD_ARGS`, `DENO_DIR`.
## Internals
### Interal: libdeno API.
Deno's privileged side will primarily be programmed in Rust. However there will
be a small C API that wraps V8 to 1) define the low-level message passing
semantics 2) provide a low-level test target 3) provide an ANSI C API binding
interface for Rust. V8 plus this C API is called libdeno and the important bits
of the API is specified here:
https://github.com/denoland/deno/blob/master/libdeno/deno.h
https://github.com/denoland/deno/blob/master/js/libdeno.ts
### Internal: Flatbuffers provide shared data between Rust and V8
We use Flatbuffers to define common structs and enums between TypeScript and
Rust. These common data structures are defined in
https://github.com/denoland/deno/blob/master/src/msg.fbs
## Contributing
See
[CONTRIBUTING.md](https://github.com/denoland/deno/blob/master/.github/CONTRIBUTING.md).
## Change Log
### 2018.10.18 / v0.1.8 / Connecting to Tokio / Fleshing out APIs
Most file system ops were implemented. Basic TCP networking is implemented.
Basic stdio streams exposed. And many random OS facilities were exposed (e.g.
environmental variables)
Tokio was chosen as the backing event loop library. A careful mapping of JS
Promises onto Rust Futures was made, preserving error handling and the ability
to execute synchronously in the main thread.
Continuous benchmarks were added: https://denoland.github.io/deno/ Performance
issues are beginning to be addressed.
"deno --types" was added to reference runtime APIs.
Working towards https://github.com/denoland/deno/milestone/2 We expect v0.2 to
be released in last October or early November.
### 2018.09.09 / v0.1.3 / Scale binding infrastructure
ETA v.0.2 October 2018 https://github.com/denoland/deno/milestone/2
We decided to use Tokio https://tokio.rs/ to provide asynchronous I/O, thread
pool execution, and as a base for high level support for various internet
protocols like HTTP. Tokio is strongly designed around the idea of Futures -
which map quite well onto JavaScript promises. We want to make it as easy as
possible to start a Tokio future from JavaScript and get a Promise for handling
it. We expect this to result in preliminary file system operations, fetch() for
http. Additionally we are working on CI, release, and benchmarking
infrastructure to scale development.
### 2018.08.23 / v0.1.0 / Rust rewrite / V8 snapshot
https://github.com/denoland/deno/commit/68d388229ea6ada339d68eb3d67feaff7a31ca97
Complete! https://github.com/denoland/deno/milestone/1
Go is a garbage collected language and we are worried that combining it with
V8's GC will lead to difficult contention problems down the road.
The V8Worker2 binding/concept is being ported to a new C++ library called
libdeno. libdeno will include the entire JS runtime as a V8 snapshot. It still
follows the message passing paradigm. Rust will be bound to this library to
implement the privileged part of Deno. See deno2/README.md for more details.
V8 Snapshots allow Deno to avoid recompiling the TypeScript compiler at startup.
This is already working.
When the rewrite is at feature parity with the Go prototype, we will release
binaries for people to try.
### 2018.09.32 / v0.0.0 / Golang Prototype / JSConf talk
https://github.com/denoland/deno/tree/golang
https://www.youtube.com/watch?v=M3BM9TB-8yA
http://tinyclouds.org/jsconf2018.pdf
### 2007-2017 / Prehistory
https://github.com/ry/v8worker
http://libuv.org/
http://tinyclouds.org/iocp-links.html
https://nodejs.org/
https://github.com/nodejs/http-parser
http://tinyclouds.org/libebb/
https://en.wikipedia.org/wiki/Merb

View file

@ -52,13 +52,13 @@
## Install
**With Python:**
With Python:
```
curl -sSf https://raw.githubusercontent.com/denoland/deno_install/master/install.py | python
```
**With PowerShell:**
With PowerShell:
```powershell
iex (iwr https://raw.githubusercontent.com/denoland/deno_install/master/install.ps1)
@ -82,89 +82,7 @@ Under development.
We make binary releases [here](https://github.com/denoland/deno/releases).
Progress towards future releases is tracked
[here](https://github.com/denoland/deno/milestones).
Roadmap is [here](https://github.com/denoland/deno/blob/master/Roadmap.md). Also
see [this presentation](http://tinyclouds.org/jsconf2018.pdf).
[Benchmarks](https://denoland.github.io/deno/)
[Chat room](https://gitter.im/denolife/Lobby).
## Build instructions
To ensure reproducible builds, Deno has most of its dependencies in a git
submodule. However, you need to install separately:
1. [Rust](https://www.rust-lang.org/en-US/install.html)
2. [Node](http://nodejs.org/)
3. Python 2.
[Not 3](https://github.com/denoland/deno/issues/464#issuecomment-411795578).
4. [ccache](https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/ccache)
(Optional but helpful for speeding up rebuilds of V8.).
5. Extra steps for Windows users:
1. Add `python.exe` to `PATH`. E.g. `set PATH=%PATH%;C:\Python27\python.exe`
2. Get [VS Community 2017](https://www.visualstudio.com/downloads/), make
sure to select the option to install C++ tools and the Windows SDK
3. Enable `Debugging Tools for Windows`, Goto Control Panel -> Windows 10 SDK
-> Right-Click -> Change -> Change -> Check Debugging Tools for Windows ->
Change -> Finish
#### To build:
# Fetch deps.
git clone --recurse-submodules https://github.com/denoland/deno.git
cd deno
./tools/setup.py
# Build.
./tools/build.py
# Run.
./out/debug/deno tests/002_hello.ts
# Test.
./tools/test.py
# Format code.
./tools/format.py
Before running `./tools/format.py`, make sure you have `yapf` installed for the
current Python and `rustfmt` for Rust. They can be installed by:
pip install yapf
rustup component add rustfmt-preview
Other useful commands:
# Call ninja manually.
./third_party/depot_tools/ninja -C out/debug
# Build a release binary.
DENO_BUILD_MODE=release ./tools/build.py :deno
# List executable targets.
./third_party/depot_tools/gn ls out/debug //:* --as=output --type=executable
# List build configuation.
./third_party/depot_tools/gn args out/debug/ --list
# Edit build configuration.
./third_party/depot_tools/gn args out/debug/
# Describe a target.
./third_party/depot_tools/gn desc out/debug/ :deno
./third_party/depot_tools/gn help
Env vars: `DENO_BUILD_MODE`, `DENO_BUILD_PATH`, `DENO_BUILD_ARGS`, `DENO_DIR`.
## Contributing
1. Fork [this repository](https://github.com/denoland/deno) and create your
branch from `master`.
2. Make your change.
3. Ensure `./tools/test.py` passes.
4. Format your code with `./tools/format.py`.
5. Make sure `./tools/lint.py` passes.
6. Send a pull request.
7. Sign the [CLA](https://cla-assistant.io/denoland/deno), if you haven't
already.
Docs are [here](https://github.com/denoland/deno/blob/master/Docs.md).
<!-- prettier-ignore -->
[avy badge]: https://ci.appveyor.com/api/projects/status/yel7wtcqwoy0to8x?branch=master&svg=true

View file

@ -2,44 +2,6 @@
API and Feature requests should be submitted as PRs to this document.
## Target Use Cases
### Implementation of `cat`
[#721](https://github.com/denoland/deno/issues/721)
```ts
import * as deno from "deno";
for (let i = 1; i < deno.argv.length; i++) {
let filename = deno.argv[i];
let file = await deno.open(filename);
await deno.copy(deno.stdout, file);
}
```
### TCP Server
[#725](https://github.com/denoland/deno/issues/725)
```ts
import * as deno from "deno";
const listener = deno.listen("tcp", ":8080");
for await (const conn of listener.accept()) {
deno.copy(conn, conn);
}
```
### List deps (implemented)
```
% deno --deps http://gist.com/blah.js
http://gist.com/blah.js
http://gist.com/dep.js
https://github.com/denoland/deno/master/testing.js
%
```
## Security Model (partially implemented)
- We want to be secure by default; user should be able to run untrusted code,
@ -78,126 +40,7 @@ Program requests to spawn `rm -rf /`. Grant? [yNs]
- in version two we will add ability to give finer grain access
--allow-net=facebook.com
## Milestone 1: Rust rewrite / V8 snapshot
Complete! https://github.com/denoland/deno/milestone/1
Go is a garbage collected language and we are worried that combining it with
V8's GC will lead to difficult contention problems down the road.
The V8Worker2 binding/concept is being ported to a new C++ library called
libdeno. libdeno will include the entire JS runtime as a V8 snapshot. It still
follows the message passing paradigm. Rust will be bound to this library to
implement the privileged part of Deno. See deno2/README.md for more details.
V8 Snapshots allow Deno to avoid recompiling the TypeScript compiler at startup.
This is already working.
When the rewrite is at feature parity with the Go prototype, we will release
binaries for people to try.
## Milestone 2: Scale binding infrastructure
ETA: October 2018 https://github.com/denoland/deno/milestone/2
We decided to use Tokio https://tokio.rs/ to provide asynchronous I/O, thread
pool execution, and as a base for high level support for various internet
protocols like HTTP. Tokio is strongly designed around the idea of Futures -
which map quite well onto JavaScript promises. We want to make it as easy as
possible to start a Tokio future from JavaScript and get a Promise for handling
it. We expect this to result in preliminary file system operations, fetch() for
http. Additionally we are working on CI, release, and benchmarking
infrastructure to scale development.
## libdeno C API.
Deno's privileged side will primarily be programmed in Rust. However there will
be a small C API that wraps V8 to 1) define the low-level message passing
semantics 2) provide a low-level test target 3) provide an ANSI C API binding
interface for Rust. V8 plus this C API is called libdeno and the important bits
of the API is specified here:
```c
// Data that gets transmitted.
typedef struct {
const char* data;
size_t len;
} deno_buf;
typedef void (*deno_sub_cb)(Deno* d, deno_buf bufs[], size_t nbufs)
void deno_set_callback(Deno* deno, deno_sub_cb cb);
// Executes javascript source code.
// Get error text with deno_last_exception().
// 0 = success, non-zero = failure.
// TODO(ry) Currently the return code has opposite semantics.
int deno_execute(Deno* d, void* user_data, const char* js_filename, const char* js_source);
// This call doesn't go into JS. This is thread-safe.
// TODO(ry) Currently this is called deno_pub. It should be renamed.
// deno_append is the desired name.
void deno_append(deno_buf buf);
// Should only be called at most once during the deno_sub_cb.
void deno_set_response(Deno* deno, deno_buf bufs[], size_t nbufs);
const char* deno_last_exception(Deno* d);
```
## TypeScript API.
This section will not attempt to over all of the APIs but give a general sense
of them.
### Internal: libdeno
This is the lowest-level interface to the privileged side. It provides little
more than passing ArrayBuffers in and out of the VM. The libdeno API is more or
less feature complete now. See
https://github.com/denoland/deno/blob/master/js/libdeno.ts
### Internal: Shared data between Rust and V8
We use Flatbuffers to define common structs and enums between TypeScript and
Rust. These common data structures are defined in
https://github.com/denoland/deno/blob/master/src/msg.fbs This is more or less
working.
### Public API
This is the global variables and various built-in modules, namely the `"deno"`
module.
Deno will provide common browser global utilities like `fetch()` and
`setTimeout()`.
Deno has typescript built-in. Users can access the built-in typescript using:
```ts
import * as ts from "typescript";
```
Deno has its own built-in module which is imported with:
```ts
import * as deno from "deno";
```
The rest of this section discusses what will be in the `deno` module.
Within Deno this is the high-level user facing API. However, the intention is to
expose functionality as simply as possible. There should be little or no
"ergonomics" APIs. (For example, `deno.readFileSync` only deals with
ArrayBuffers and does not have an encoding parameter to return strings.) The
intention is to make very easy to extend and link in external modules which can
then add this functionality.
Deno does not aim to be API compatible with Node in any respect. Deno will
export a single flat namespace "deno" under which all core functions are
defined. We leave it up to users to wrap Deno's namespace to provide some
compatibility with Node.
#### Top-level Await (Not Implemented)
## Top-level Await (Not Implemented)
[#471](https://github.com/denoland/deno/issues/471)
@ -205,162 +48,14 @@ This will be put off until at least deno2 Milestone1 is complete. One of the
major problems is that top-level await calls are not syntactically valid
TypeScript.
#### I/O (Not Implemented) [#721](https://github.com/denoland/deno/issues/721)
### [Broken] List dependencies of a program.
There are many OS constructs that perform I/O: files, sockets, pipes. Deno aims
to provide a unified lowest common denominator interface to work with these
objects. Deno needs to operate on all of these asynchronously in order to not
block the event loop and it.
Currently broken: https://github.com/denoland/deno/issues/1011
Sockets and pipes support non-blocking reads and write. Generally file I/O is
blocking but it can be done in a thread pool to avoid blocking the main thread.
Although file I/O can be made asynchronous, it does not support the same
non-blocking reads and writes that sockets and pipes do.
The following interfaces support files, socket, and pipes and are heavily
inspired by Go. The main difference in porting to JavaScript is that errors will
be handled by exceptions, modulo EOF, which is returned as part of `ReadResult`.
```ts
// The bytes read during an I/O call and a boolean indicating EOF.
interface ReadResult {
nread: number;
eof: boolean;
}
// Reader is the interface that wraps the basic read() method.
// https://golang.org/pkg/io/#Reader
interface Reader {
// read() reads up to p.byteLength bytes into p. It returns the number of bytes
// read (0 <= n <= p.byteLength) and any error encountered. Even if read()
// returns n < p.byteLength, it may use all of p as scratch space during the
// call. If some data is available but not p.byteLength bytes, read()
// conventionally returns what is available instead of waiting for more.
//
// When read() encounters an error or end-of-file condition after successfully
// reading n > 0 bytes, it returns the number of bytes read. It may return the
// (non-nil) error from the same call or return the error (and n == 0) from a
// subsequent call. An instance of this general case is that a Reader
// returning a non-zero number of bytes at the end of the input stream may
// return either err == EOF or err == nil. The next read() should return 0, EOF.
//
// Callers should always process the n > 0 bytes returned before considering
// the error err. Doing so correctly handles I/O errors that happen after
// reading some bytes and also both of the allowed EOF behaviors.
//
// Implementations of read() are discouraged from returning a zero byte count
// with a nil error, except when p.byteLength == 0. Callers should treat a
// return of 0 and nil as indicating that nothing happened; in particular it
// does not indicate EOF.
//
// Implementations must not retain p.
read(p: ArrayBufferView): Promise<ReadResult>;
}
// Writer is the interface that wraps the basic write() method.
// https://golang.org/pkg/io/#Writer
interface Writer {
// write() writes p.byteLength bytes from p to the underlying data stream. It
// returns the number of bytes written from p (0 <= n <= p.byteLength) and any
// error encountered that caused the write to stop early. write() must return a
// non-nil error if it returns n < p.byteLength. write() must not modify the
// slice data, even temporarily.
//
// Implementations must not retain p.
write(p: ArrayBufferView): Promise<number>;
}
// https://golang.org/pkg/io/#Closer
interface Closer {
// The behavior of Close after the first call is undefined. Specific
// implementations may document their own behavior.
close(): void;
}
// https://golang.org/pkg/io/#Seeker
interface Seeker {
// Seek sets the offset for the next read() or write() to offset, interpreted
// according to whence: SeekStart means relative to the start of the file,
// SeekCurrent means relative to the current offset, and SeekEnd means
// relative to the end. Seek returns the new offset relative to the start of
// the file and an error, if any.
//
// Seeking to an offset before the start of the file is an error. Seeking to
// any positive offset is legal, but the behavior of subsequent I/O operations
// on the underlying object is implementation-dependent.
seek(offset: number, whence: number): Promise<void>;
}
// https://golang.org/pkg/io/#ReadCloser
interface ReaderCloser extends Reader, Closer {}
// https://golang.org/pkg/io/#WriteCloser
interface WriteCloser extends Writer, Closer {}
// https://golang.org/pkg/io/#ReadSeeker
interface ReadSeeker extends Reader, Seeker {}
// https://golang.org/pkg/io/#WriteSeeker
interface WriteSeeker extends Writer, Seeker {}
// https://golang.org/pkg/io/#ReadWriteCloser
interface ReadWriteCloser extends Reader, Writer, Closer {}
// https://golang.org/pkg/io/#ReadWriteSeeker
interface ReadWriteSeeker extends Reader, Writer, Seeker {}
```
These interfaces are well specified, simple, and have very nice utility
functions that will be easy to port. Some example utilites:
```ts
// copy() copies from src to dst until either EOF is reached on src or an error
// occurs. It returns the number of bytes copied and the first error encountered
// while copying, if any.
//
// Because copy() is defined to read from src until EOF, it does not treat an EOF
// from read() as an error to be reported.
//
// https://golang.org/pkg/io/#Copy
async function copy(dst: Writer, src: Reader): Promise<number> {
let n = 0;
const b = new ArrayBufferView(1024);
let got_eof = false;
while (got_eof === false) {
let result = await src.read(b);
if (result.eof) got_eof = true;
n += await dst.write(b.subarray(0, result.nread));
}
return n;
}
// MultiWriter creates a writer that duplicates its writes to all the provided
// writers, similar to the Unix tee(1) command.
//
// Each write is written to each listed writer, one at a time. If a listed
// writer returns an error, that overall write operation stops and returns the
// error; it does not continue down the list.
//
// https://golang.org/pkg/io/#MultiWriter
function multiWriter(writers: ...Writer): Writer {
return {
write: async (p: ArrayBufferView) => Promise<number> {
let n;
let nwritten = await Promise.all(writers.map((w) => w.write(p)));
return nwritten[0];
// TODO unsure of proper semantics for return value..
}
};
}
```
A utility function will be provided to make any `Reader` into an
`AsyncIterator`, which has very similar semanatics.
```ts
function readerIterator(r: deno.Reader): AsyncIterator<ArrayBufferView>;
// Example
for await (let buf of readerIterator(socket)) {
console.log(`read ${buf.byteLength} from socket`);
}
% deno --deps http://gist.com/blah.js
http://gist.com/blah.js
http://gist.com/dep.js
https://github.com/denoland/deno/master/testing.js
%
```

View file

@ -10,7 +10,7 @@
<p> <a href="https://github.com/denoland/deno">github.com/denoland/deno</a>
<p> <a href="https://github.com/denoland/deno/issues/573">User Documentation</a>
<p> <a href="https://github.com/denoland/deno/blob/master/Docs.md">Documentation</a>
<p> <a href="https://github.com/denoland/deno/blob/master/Roadmap.md">Roadmap</a>