Skip to Content
Reference@streamsy/core

Streams

Reference for the Streams namespace in @streamsy/core/protocol. Its functions run the stream operations on a StreamAddress over the generic protocol reader and writer.

import { Streams, StreamsReader, StreamsWriter } from "@streamsy/core/protocol";
import type * as Core from "@streamsy/core";
import type {
  AppendCallOptions,
  AppendResult,
  CreateOptions,
  CreateResult,
  HeadResult,
  ReadNextError,
  ReadNextOptions,
  ReadNextResult,
  ReadOptions,
  StreamAddress,
  StreamsFault,
} from "@streamsy/core";
import type { Array, Effect, Function, Schedule, Stream } from "effect";

Each Streams function runs a stream operation on a StreamAddress, and it needs StreamsReader, StreamsWriter, or both. The methods of a stream class run the same operations on the params of one stream. Protocol tools and tests use Streams on raw addresses. An application uses stream classes.

Most of the types of this namespace are also exported from the root of @streamsy/core. Their sections link to the Toolkit types page, which describes them.

Functions

Streams.create

declare const create: <A, RD, RE>(
  address: StreamAddress.StreamAddress<A, RD, RE>,
  options?: Omit<CreateOptions, "contentType">,
) => Effect.Effect<CreateResult, Streams.CreateFailure, StreamsWriter>;

create creates the stream of address, with the content type of the address. See CreateOptions.

Streams.append

declare const append: <A, RD, RE, const O extends AppendCallOptions = {}>(
  address: StreamAddress.StreamAddress<A, RD, RE>,
  items: Streams.AppendItems<A, O>,
  options?: O,
) => Effect.Effect<AppendResult, Streams.AppendFailure<O>, RE | StreamsWriter>;

append encodes each item with the codec of address and writes the items as one append. For a JSON address, it writes the items as one JSON array. For a bytes address, it concatenates the items into one request body.

  • append fails with EncodeFault when the codec cannot encode an item.
  • append dies with a RangeError when a bytes item encodes to zero bytes.
  • An append with no items and close: true sends an empty body, which closes the stream.
  • The error type lists only the rejections that options make possible. When the host answers with another rejection, append dies. See AppendFailure.

Streams.read

declare function read<A, RD, RE>(
  address: StreamAddress.StreamAddress<A, RD, RE>,
  options?: ReadOptions,
): Stream.Stream<Streams.Batch<A>, Streams.ReadFailure, StreamsReader | RD>;

read returns a stream of decoded batches from options.offset to the tail. It reads one page at a time, and it ends after the first batch whose upToDate is true. read fails with DecodeFault when the codec cannot decode a message. See Read.

Streams.follow

declare function follow<A, RD, RE>(
  address: StreamAddress.StreamAddress<A, RD, RE>,
  options?: Streams.FollowOptions,
): Stream.Stream<Streams.Batch<A>, Streams.FollowFailure, StreamsReader | RD>;

follow reads to the tail, then makes one long-poll wait at a time with readNext. It sends the cursor of each wait in the next one. It emits no empty batch, except the last batch of a closed stream, and it ends after a batch whose closed is true.

When a read fails with a retryable TransportFault or StorageFault, follow retries it from the same offset on the retry schedule, so it emits each item once. Any other failure ends the stream. See Follow.

Streams.items

declare const items: <A, E, R>(
  batches: Stream.Stream<Streams.Batch<A>, E, R>,
) => Stream.Stream<A, E, R>;

items flattens a stream of batches into a stream of their items, in order. Use it on the result of read or follow.

Streams.fold

declare const fold: <A, RD, RE, S>(
  address: StreamAddress.StreamAddress<A, RD, RE>,
  initial: Function.LazyArg<S>,
  f: (state: S, item: A) => S,
  options?: ReadOptions,
) => Effect.Effect<Streams.Folded<S>, Streams.ReadFailure, RD | StreamsReader>;

fold reads to the tail and folds every item into one value, starting from initial(). It returns the value and the tail that it read up to. To continue a fold from an earlier result, pass its offset as options.offset. See Fold.

Streams.collect

declare const collect: <A, RD, RE>(
  address: StreamAddress.StreamAddress<A, RD, RE>,
  options?: ReadOptions,
) => Effect.Effect<Streams.Collected<A>, Streams.ReadFailure, RD | StreamsReader>;

collect reads to the tail and returns every item, and the tail that it read up to.

Streams.decide

declare const decide: <A, RD, RE, S, B, E, R>(
  address: StreamAddress.StreamAddress<A, RD, RE>,
  spec: Streams.DecideSpec<A, S, B, E, R>,
) => Effect.Effect<
  Streams.Decided<B>,
  Streams.ReadFailure | Streams.AppendFailure<{ expectedOffset: string }> | E,
  R | RD | RE | StreamsReader | StreamsWriter
>;

decide folds the stream to its tail, calls spec.decide with the state, and appends the items of the decision with expectedOffset set to the tail that it read. When the decision has no items, decide writes nothing.

After an OffsetMismatch on its own append, decide continues the fold from the tail that it had and decides again, as long as spec.retry allows another attempt. Then it fails with the last OffsetMismatch. It retries no other failure. The rules of spec are under DecideSpec. See Read, fold, append.

Streams.session

declare const session: <A, RD, RE>(
  address: StreamAddress.StreamAddress<A, RD, RE>,
  options: ReadNextOptions,
) => Effect.Effect<ReadNextResult, StreamsFault | ReadNextError, StreamsReader>;

session makes one long-poll wait with readNext, and returns the raw result. It does not decode the messages. See Session.

Streams.head

declare const head: <A, RD, RE>(
  address: StreamAddress.StreamAddress<A, RD, RE>,
) => Effect.Effect<HeadResult, Streams.HeadFailure, StreamsReader>;

head returns the tail and the configuration of the stream, without messages.

Streams.remove

declare const remove: <A, RD, RE>(
  address: StreamAddress.StreamAddress<A, RD, RE>,
) => Effect.Effect<void, Streams.RemoveFailure, StreamsWriter>;

remove deletes the stream of address.

Options

Streams.AppendItems

type AppendItems<A, O> = O extends { readonly close: true }
  ? ReadonlyArray<A>
  : Array.NonEmptyReadonlyArray<A>;

AppendItems<A, O> is the list of items that an append takes. The list must have at least one item, unless the options O have close: true.

Streams.FollowOptions

interface FollowOptions extends ReadOptions {
  readonly retry?: Schedule.Schedule<unknown>;
}

FollowOptions holds the options of follow: the ReadOptions, and retry. retry is the schedule for a read that fails with a retryable fault. The default schedule is exponential from 200 ms with jitter, caps each wait at 30 s, and never stops.

Results

Streams.Batch

type Batch<A> = Core.Batch<A>;

Streams.Batch is the root Batch.

Streams.Collected

type Collected<A> = Core.Collected<A>;

Streams.Collected is the root Collected.

Streams.Folded

type Folded<S> = Core.Folded<S>;

Streams.Folded is the root Folded.

Streams.DecideSpec

type DecideSpec<A, S, B, E, R> = Core.DecideSpec<A, S, B, E, R>;

Streams.DecideSpec is the root DecideSpec.

Streams.Decision

type Decision<B, A> = Core.Decision<B, A>;

Streams.Decision is the root Decision.

Streams.Decided

type Decided<B> = Core.Decided<B>;

Streams.Decided is the root Decided.

Failures

Streams.ReadFailure

type ReadFailure = Core.ReadFailure;

Streams.ReadFailure is the root ReadFailure.

Streams.FollowFailure

type FollowFailure = Core.FollowFailure;

Streams.FollowFailure is the root FollowFailure.

Streams.AppendFailure

type AppendFailure<O = Core.AppendCallOptions> = Core.AppendFailure<O>;

Streams.AppendFailure is the root AppendFailure.

Streams.CreateFailure

type CreateFailure = Core.CreateFailure;

Streams.CreateFailure is the root CreateFailure.

Streams.HeadFailure

type HeadFailure = Core.HeadFailure;

Streams.HeadFailure is the root HeadFailure.

Streams.RemoveFailure

type RemoveFailure = Core.RemoveFailure;

Streams.RemoveFailure is the root RemoveFailure.

On this page