1
0
Fork 0
mirror of https://github.com/denoland/deno.git synced 2024-12-29 10:39:10 -05:00
denoland-deno/core/io.rs

301 lines
7.4 KiB
Rust
Raw Normal View History

// Copyright 2018-2023 the Deno authors. All rights reserved. MIT license.
feat(core): improve resource read & write traits (#16115) This commit introduces two new buffer wrapper types to `deno_core`. The main benefit of these new wrappers is that they can wrap a number of different underlying buffer types. This allows for a more flexible read and write API on resources that will require less copying of data between different buffer representations. - `BufView` is a read-only view onto a buffer. It can be backed by `ZeroCopyBuf`, `Vec<u8>`, and `bytes::Bytes`. - `BufViewMut` is a read-write view onto a buffer. It can be cheaply converted into a `BufView`. It can be backed by `ZeroCopyBuf` or `Vec<u8>`. Both new buffer views have a cursor. This means that the start point of the view can be constrained to write / read from just a slice of the view. Only the start point of the slice can be adjusted. The end point is fixed. To adjust the end point, the underlying buffer needs to be truncated. Readable resources have been changed to better cater to resources that do not support BYOB reads. The basic `read` method now returns a `BufView` instead of taking a `ZeroCopyBuf` to fill. This allows the operation to return buffers that the resource has already allocated, instead of forcing the caller to allocate the buffer. BYOB reads are still very useful for resources that support them, so a new `read_byob` method has been added that takes a `BufViewMut` to fill. `op_read` attempts to use `read_byob` if the resource supports it, which falls back to `read` and performs an additional copy if it does not. For Rust->JS reads this change should have no impact, but for Rust->Rust reads, this allows the caller to avoid an additional copy in many scenarios. This combined with the support for `BufView` to be backed by `bytes::Bytes` allows us to avoid one data copy when piping from a `fetch` response into an `ext/http` response. Writable resources have been changed to take a `BufView` instead of a `ZeroCopyBuf` as an argument. This allows for less copying of data in certain scenarios, as described above. Additionally a new `Resource::write_all` method has been added that takes a `BufView` and continually attempts to write the resource until the entire buffer has been written. Certain resources like files can override this method to provide a more efficient `write_all` implementation.
2022-10-09 10:49:25 -04:00
use std::ops::Deref;
use std::ops::DerefMut;
use bytes::Buf;
feat(core): improve resource read & write traits (#16115) This commit introduces two new buffer wrapper types to `deno_core`. The main benefit of these new wrappers is that they can wrap a number of different underlying buffer types. This allows for a more flexible read and write API on resources that will require less copying of data between different buffer representations. - `BufView` is a read-only view onto a buffer. It can be backed by `ZeroCopyBuf`, `Vec<u8>`, and `bytes::Bytes`. - `BufViewMut` is a read-write view onto a buffer. It can be cheaply converted into a `BufView`. It can be backed by `ZeroCopyBuf` or `Vec<u8>`. Both new buffer views have a cursor. This means that the start point of the view can be constrained to write / read from just a slice of the view. Only the start point of the slice can be adjusted. The end point is fixed. To adjust the end point, the underlying buffer needs to be truncated. Readable resources have been changed to better cater to resources that do not support BYOB reads. The basic `read` method now returns a `BufView` instead of taking a `ZeroCopyBuf` to fill. This allows the operation to return buffers that the resource has already allocated, instead of forcing the caller to allocate the buffer. BYOB reads are still very useful for resources that support them, so a new `read_byob` method has been added that takes a `BufViewMut` to fill. `op_read` attempts to use `read_byob` if the resource supports it, which falls back to `read` and performs an additional copy if it does not. For Rust->JS reads this change should have no impact, but for Rust->Rust reads, this allows the caller to avoid an additional copy in many scenarios. This combined with the support for `BufView` to be backed by `bytes::Bytes` allows us to avoid one data copy when piping from a `fetch` response into an `ext/http` response. Writable resources have been changed to take a `BufView` instead of a `ZeroCopyBuf` as an argument. This allows for less copying of data in certain scenarios, as described above. Additionally a new `Resource::write_all` method has been added that takes a `BufView` and continually attempts to write the resource until the entire buffer has been written. Certain resources like files can override this method to provide a more efficient `write_all` implementation.
2022-10-09 10:49:25 -04:00
use serde_v8::ZeroCopyBuf;
/// BufView is a wrapper around an underlying contiguous chunk of bytes. It can
/// be created from a [ZeroCopyBuf], [bytes::Bytes], or [Vec<u8>] and implements
/// `Deref<[u8]>` and `AsRef<[u8]>`.
///
/// The wrapper has the ability to constrain the exposed view to a sub-region of
/// the underlying buffer. This is useful for write operations, because they may
/// have to be called multiple times, with different views onto the buffer to be
/// able to write it entirely.
pub struct BufView {
inner: BufViewInner,
cursor: usize,
}
enum BufViewInner {
Empty,
Bytes(bytes::Bytes),
ZeroCopy(ZeroCopyBuf),
Vec(Vec<u8>),
}
impl BufView {
const fn from_inner(inner: BufViewInner) -> Self {
feat(core): improve resource read & write traits (#16115) This commit introduces two new buffer wrapper types to `deno_core`. The main benefit of these new wrappers is that they can wrap a number of different underlying buffer types. This allows for a more flexible read and write API on resources that will require less copying of data between different buffer representations. - `BufView` is a read-only view onto a buffer. It can be backed by `ZeroCopyBuf`, `Vec<u8>`, and `bytes::Bytes`. - `BufViewMut` is a read-write view onto a buffer. It can be cheaply converted into a `BufView`. It can be backed by `ZeroCopyBuf` or `Vec<u8>`. Both new buffer views have a cursor. This means that the start point of the view can be constrained to write / read from just a slice of the view. Only the start point of the slice can be adjusted. The end point is fixed. To adjust the end point, the underlying buffer needs to be truncated. Readable resources have been changed to better cater to resources that do not support BYOB reads. The basic `read` method now returns a `BufView` instead of taking a `ZeroCopyBuf` to fill. This allows the operation to return buffers that the resource has already allocated, instead of forcing the caller to allocate the buffer. BYOB reads are still very useful for resources that support them, so a new `read_byob` method has been added that takes a `BufViewMut` to fill. `op_read` attempts to use `read_byob` if the resource supports it, which falls back to `read` and performs an additional copy if it does not. For Rust->JS reads this change should have no impact, but for Rust->Rust reads, this allows the caller to avoid an additional copy in many scenarios. This combined with the support for `BufView` to be backed by `bytes::Bytes` allows us to avoid one data copy when piping from a `fetch` response into an `ext/http` response. Writable resources have been changed to take a `BufView` instead of a `ZeroCopyBuf` as an argument. This allows for less copying of data in certain scenarios, as described above. Additionally a new `Resource::write_all` method has been added that takes a `BufView` and continually attempts to write the resource until the entire buffer has been written. Certain resources like files can override this method to provide a more efficient `write_all` implementation.
2022-10-09 10:49:25 -04:00
Self { inner, cursor: 0 }
}
pub const fn empty() -> Self {
feat(core): improve resource read & write traits (#16115) This commit introduces two new buffer wrapper types to `deno_core`. The main benefit of these new wrappers is that they can wrap a number of different underlying buffer types. This allows for a more flexible read and write API on resources that will require less copying of data between different buffer representations. - `BufView` is a read-only view onto a buffer. It can be backed by `ZeroCopyBuf`, `Vec<u8>`, and `bytes::Bytes`. - `BufViewMut` is a read-write view onto a buffer. It can be cheaply converted into a `BufView`. It can be backed by `ZeroCopyBuf` or `Vec<u8>`. Both new buffer views have a cursor. This means that the start point of the view can be constrained to write / read from just a slice of the view. Only the start point of the slice can be adjusted. The end point is fixed. To adjust the end point, the underlying buffer needs to be truncated. Readable resources have been changed to better cater to resources that do not support BYOB reads. The basic `read` method now returns a `BufView` instead of taking a `ZeroCopyBuf` to fill. This allows the operation to return buffers that the resource has already allocated, instead of forcing the caller to allocate the buffer. BYOB reads are still very useful for resources that support them, so a new `read_byob` method has been added that takes a `BufViewMut` to fill. `op_read` attempts to use `read_byob` if the resource supports it, which falls back to `read` and performs an additional copy if it does not. For Rust->JS reads this change should have no impact, but for Rust->Rust reads, this allows the caller to avoid an additional copy in many scenarios. This combined with the support for `BufView` to be backed by `bytes::Bytes` allows us to avoid one data copy when piping from a `fetch` response into an `ext/http` response. Writable resources have been changed to take a `BufView` instead of a `ZeroCopyBuf` as an argument. This allows for less copying of data in certain scenarios, as described above. Additionally a new `Resource::write_all` method has been added that takes a `BufView` and continually attempts to write the resource until the entire buffer has been written. Certain resources like files can override this method to provide a more efficient `write_all` implementation.
2022-10-09 10:49:25 -04:00
Self::from_inner(BufViewInner::Empty)
}
/// Get the length of the buffer view. This is the length of the underlying
/// buffer minus the cursor position.
pub fn len(&self) -> usize {
match &self.inner {
BufViewInner::Empty => 0,
BufViewInner::Bytes(bytes) => bytes.len() - self.cursor,
BufViewInner::ZeroCopy(zero_copy) => zero_copy.len() - self.cursor,
BufViewInner::Vec(vec) => vec.len() - self.cursor,
}
}
/// Is the buffer view empty?
pub fn is_empty(&self) -> bool {
self.len() == 0
}
/// Advance the internal cursor of the buffer view by `n` bytes.
pub fn advance_cursor(&mut self, n: usize) {
assert!(self.len() >= n);
self.cursor += n;
}
/// Reset the internal cursor of the buffer view to the beginning of the
/// buffer. Returns the old cursor position.
pub fn reset_cursor(&mut self) -> usize {
let old = self.cursor;
self.cursor = 0;
old
}
}
impl Buf for BufView {
fn remaining(&self) -> usize {
self.len()
}
fn chunk(&self) -> &[u8] {
self.deref()
}
fn advance(&mut self, cnt: usize) {
self.advance_cursor(cnt)
}
}
feat(core): improve resource read & write traits (#16115) This commit introduces two new buffer wrapper types to `deno_core`. The main benefit of these new wrappers is that they can wrap a number of different underlying buffer types. This allows for a more flexible read and write API on resources that will require less copying of data between different buffer representations. - `BufView` is a read-only view onto a buffer. It can be backed by `ZeroCopyBuf`, `Vec<u8>`, and `bytes::Bytes`. - `BufViewMut` is a read-write view onto a buffer. It can be cheaply converted into a `BufView`. It can be backed by `ZeroCopyBuf` or `Vec<u8>`. Both new buffer views have a cursor. This means that the start point of the view can be constrained to write / read from just a slice of the view. Only the start point of the slice can be adjusted. The end point is fixed. To adjust the end point, the underlying buffer needs to be truncated. Readable resources have been changed to better cater to resources that do not support BYOB reads. The basic `read` method now returns a `BufView` instead of taking a `ZeroCopyBuf` to fill. This allows the operation to return buffers that the resource has already allocated, instead of forcing the caller to allocate the buffer. BYOB reads are still very useful for resources that support them, so a new `read_byob` method has been added that takes a `BufViewMut` to fill. `op_read` attempts to use `read_byob` if the resource supports it, which falls back to `read` and performs an additional copy if it does not. For Rust->JS reads this change should have no impact, but for Rust->Rust reads, this allows the caller to avoid an additional copy in many scenarios. This combined with the support for `BufView` to be backed by `bytes::Bytes` allows us to avoid one data copy when piping from a `fetch` response into an `ext/http` response. Writable resources have been changed to take a `BufView` instead of a `ZeroCopyBuf` as an argument. This allows for less copying of data in certain scenarios, as described above. Additionally a new `Resource::write_all` method has been added that takes a `BufView` and continually attempts to write the resource until the entire buffer has been written. Certain resources like files can override this method to provide a more efficient `write_all` implementation.
2022-10-09 10:49:25 -04:00
impl Deref for BufView {
type Target = [u8];
fn deref(&self) -> &[u8] {
let buf = match &self.inner {
BufViewInner::Empty => &[],
BufViewInner::Bytes(bytes) => bytes.deref(),
BufViewInner::ZeroCopy(zero_copy) => zero_copy.deref(),
BufViewInner::Vec(vec) => vec.deref(),
};
&buf[self.cursor..]
}
}
impl AsRef<[u8]> for BufView {
fn as_ref(&self) -> &[u8] {
self.deref()
}
}
impl From<ZeroCopyBuf> for BufView {
fn from(buf: ZeroCopyBuf) -> Self {
Self::from_inner(BufViewInner::ZeroCopy(buf))
}
}
impl From<Vec<u8>> for BufView {
fn from(vec: Vec<u8>) -> Self {
Self::from_inner(BufViewInner::Vec(vec))
}
}
impl From<bytes::Bytes> for BufView {
fn from(buf: bytes::Bytes) -> Self {
Self::from_inner(BufViewInner::Bytes(buf))
}
}
impl From<BufView> for bytes::Bytes {
fn from(buf: BufView) -> Self {
match buf.inner {
BufViewInner::Empty => bytes::Bytes::new(),
BufViewInner::Bytes(bytes) => bytes,
BufViewInner::ZeroCopy(zero_copy) => zero_copy.into(),
BufViewInner::Vec(vec) => vec.into(),
}
}
}
/// BufMutView is a wrapper around an underlying contiguous chunk of writable
/// bytes. It can be created from a `ZeroCopyBuf` or a `Vec<u8>` and implements
/// `DerefMut<[u8]>` and `AsMut<[u8]>`.
///
/// The wrapper has the ability to constrain the exposed view to a sub-region of
/// the underlying buffer. This is useful for write operations, because they may
/// have to be called multiple times, with different views onto the buffer to be
/// able to write it entirely.
///
/// A `BufMutView` can be turned into a `BufView` by calling `BufMutView::into_view`.
pub struct BufMutView {
inner: BufMutViewInner,
cursor: usize,
}
enum BufMutViewInner {
ZeroCopy(ZeroCopyBuf),
Vec(Vec<u8>),
}
impl BufMutView {
fn from_inner(inner: BufMutViewInner) -> Self {
Self { inner, cursor: 0 }
}
pub fn new(len: usize) -> Self {
Self::from_inner(BufMutViewInner::Vec(vec![0; len]))
}
/// Get the length of the buffer view. This is the length of the underlying
/// buffer minus the cursor position.
pub fn len(&self) -> usize {
match &self.inner {
BufMutViewInner::ZeroCopy(zero_copy) => zero_copy.len() - self.cursor,
BufMutViewInner::Vec(vec) => vec.len() - self.cursor,
}
}
/// Is the buffer view empty?
pub fn is_empty(&self) -> bool {
self.len() == 0
}
/// Advance the internal cursor of the buffer view by `n` bytes.
pub fn advance_cursor(&mut self, n: usize) {
assert!(self.len() >= n);
self.cursor += n;
}
/// Reset the internal cursor of the buffer view to the beginning of the
/// buffer. Returns the old cursor position.
pub fn reset_cursor(&mut self) -> usize {
let old = self.cursor;
self.cursor = 0;
old
}
/// Turn this `BufMutView` into a `BufView`.
pub fn into_view(self) -> BufView {
let inner = match self.inner {
BufMutViewInner::ZeroCopy(zero_copy) => BufViewInner::ZeroCopy(zero_copy),
BufMutViewInner::Vec(vec) => BufViewInner::Vec(vec),
};
BufView {
inner,
cursor: self.cursor,
}
}
/// Unwrap the underlying buffer into a `Vec<u8>`, consuming the `BufMutView`.
///
/// This method panics when called on a `BufMutView` that was created from a
/// `ZeroCopyBuf`.
pub fn unwrap_vec(self) -> Vec<u8> {
match self.inner {
BufMutViewInner::ZeroCopy(_) => {
panic!("Cannot unwrap a ZeroCopyBuf backed BufMutView into a Vec");
}
BufMutViewInner::Vec(vec) => vec,
}
}
/// Get a mutable reference to an underlying `Vec<u8>`.
///
/// This method panics when called on a `BufMutView` that was created from a
/// `ZeroCopyBuf`.
pub fn get_mut_vec(&mut self) -> &mut Vec<u8> {
match &mut self.inner {
BufMutViewInner::ZeroCopy(_) => {
panic!("Cannot unwrap a ZeroCopyBuf backed BufMutView into a Vec");
}
BufMutViewInner::Vec(vec) => vec,
}
}
}
impl Buf for BufMutView {
fn remaining(&self) -> usize {
self.len()
}
fn chunk(&self) -> &[u8] {
self.deref()
}
fn advance(&mut self, cnt: usize) {
self.advance_cursor(cnt)
}
}
feat(core): improve resource read & write traits (#16115) This commit introduces two new buffer wrapper types to `deno_core`. The main benefit of these new wrappers is that they can wrap a number of different underlying buffer types. This allows for a more flexible read and write API on resources that will require less copying of data between different buffer representations. - `BufView` is a read-only view onto a buffer. It can be backed by `ZeroCopyBuf`, `Vec<u8>`, and `bytes::Bytes`. - `BufViewMut` is a read-write view onto a buffer. It can be cheaply converted into a `BufView`. It can be backed by `ZeroCopyBuf` or `Vec<u8>`. Both new buffer views have a cursor. This means that the start point of the view can be constrained to write / read from just a slice of the view. Only the start point of the slice can be adjusted. The end point is fixed. To adjust the end point, the underlying buffer needs to be truncated. Readable resources have been changed to better cater to resources that do not support BYOB reads. The basic `read` method now returns a `BufView` instead of taking a `ZeroCopyBuf` to fill. This allows the operation to return buffers that the resource has already allocated, instead of forcing the caller to allocate the buffer. BYOB reads are still very useful for resources that support them, so a new `read_byob` method has been added that takes a `BufViewMut` to fill. `op_read` attempts to use `read_byob` if the resource supports it, which falls back to `read` and performs an additional copy if it does not. For Rust->JS reads this change should have no impact, but for Rust->Rust reads, this allows the caller to avoid an additional copy in many scenarios. This combined with the support for `BufView` to be backed by `bytes::Bytes` allows us to avoid one data copy when piping from a `fetch` response into an `ext/http` response. Writable resources have been changed to take a `BufView` instead of a `ZeroCopyBuf` as an argument. This allows for less copying of data in certain scenarios, as described above. Additionally a new `Resource::write_all` method has been added that takes a `BufView` and continually attempts to write the resource until the entire buffer has been written. Certain resources like files can override this method to provide a more efficient `write_all` implementation.
2022-10-09 10:49:25 -04:00
impl Deref for BufMutView {
type Target = [u8];
fn deref(&self) -> &[u8] {
let buf = match &self.inner {
BufMutViewInner::ZeroCopy(zero_copy) => zero_copy.deref(),
BufMutViewInner::Vec(vec) => vec.deref(),
};
&buf[self.cursor..]
}
}
impl DerefMut for BufMutView {
fn deref_mut(&mut self) -> &mut [u8] {
let buf = match &mut self.inner {
BufMutViewInner::ZeroCopy(zero_copy) => zero_copy.deref_mut(),
BufMutViewInner::Vec(vec) => vec.deref_mut(),
};
&mut buf[self.cursor..]
}
}
impl AsRef<[u8]> for BufMutView {
fn as_ref(&self) -> &[u8] {
self.deref()
}
}
impl AsMut<[u8]> for BufMutView {
fn as_mut(&mut self) -> &mut [u8] {
self.deref_mut()
}
}
impl From<ZeroCopyBuf> for BufMutView {
fn from(buf: ZeroCopyBuf) -> Self {
Self::from_inner(BufMutViewInner::ZeroCopy(buf))
}
}
impl From<Vec<u8>> for BufMutView {
fn from(buf: Vec<u8>) -> Self {
Self::from_inner(BufMutViewInner::Vec(buf))
}
}
pub enum WriteOutcome {
Partial { nwritten: usize, view: BufView },
Full { nwritten: usize },
}
impl WriteOutcome {
pub fn nwritten(&self) -> usize {
match self {
WriteOutcome::Partial { nwritten, .. } => *nwritten,
WriteOutcome::Full { nwritten } => *nwritten,
}
}
}