Skip to Content
Reference@streamsy/core

Streamsy

Reference for the Streamsy namespace in @streamsy/core. It declares stream classes and groups them for a host.

import { Streamsy, StreamAddress } from "@streamsy/core";
import type {
  AppendCallOptions,
  AppendResult,
  CreateOptions,
  CreateResult,
  DecodeFault,
  HeadResult,
  ReadNextOptions,
  ReadNextResult,
  ReadOptions,
  StreamsFault,
} from "@streamsy/core";
import type { Reader, StreamRoute, Streams, Writer } from "@streamsy/core/protocol";
import type { Context, Effect, Function, Option, Schema, Stream } from "effect";

A stream class declares a set of streams with a service name, an id template, and a codec. A group lists the stream classes that a host stores and serves. For a worked example, see Declare streams.

The signatures use StreamRoute from @streamsy/core/protocol. StreamRoute.ParamCodecs is a record with one Schema.Codec<unknown, string> for each template param. StreamRoute.Params<C> is the record of decoded params. StreamRoute.ExactTemplateParams makes the compiler check that the template and the record name the same params.

Streamsy.json

declare function json<Self>(): <
  const Name extends string,
  const Template extends string,
  Codecs extends StreamRoute.ParamCodecs,
  A,
  I,
  RD,
  RE,
>(
  name: Name,
  template: Template,
  options: {
    readonly params: Codecs & StreamRoute.ExactTemplateParams<Template, Codecs>;
    readonly schema: Schema.Codec<A, I, RD, RE>;
  },
) => Streamsy.JsonClass<Self, Name, Template, Codecs, A, I, RD, RE>;

json declares a stream class whose items are values of schema, stored as JSON with the content type application/json. The server stores each item of an append as a separate message.

json takes the class type first and the arguments second. Call it twice and extend the result:

import { Schema } from "effect";
import { Streamsy } from "@streamsy/core";

const Note = Schema.Struct({ id: Schema.String, text: Schema.String });

class Drafts extends Streamsy.json<Drafts>()("Notes/Drafts", "drafts/:draftId", {
  params: { draftId: Schema.String },
  schema: Note,
}) {}

Each argument has its own rules:

  • name is the global name of the service, and the service key is @streamsy/core/Stream/<name>. Two classes with the same name are the same service, so prefix each name with an application namespace, as in Notes/Drafts. The name Wildcard is reserved.
  • template is a relative path in which a :name segment is a param, for example drafts/:draftId.
  • params has one codec for each :name in the template, and no other codec. The compiler checks this when template is a string literal. The call also checks it at run time and throws a RangeError on a mismatch, which catches a template typed as a plain string.

Streamsy.bytes

declare function bytes<Self>(): <
  const Name extends string,
  const Template extends string,
  Codecs extends StreamRoute.ParamCodecs,
>(
  name: Name,
  template: Template,
  options: {
    readonly params: Codecs & StreamRoute.ExactTemplateParams<Template, Codecs>;
    readonly contentType?: string;
  },
) => Streamsy.BytesClass<Self, Name, Template, Codecs>;

bytes declares a stream class whose items are Uint8Array values. An append concatenates its items into one request body, and the server stores that body as one message. The default content type is application/octet-stream.

The rules for name, template, and params are the same as for Streamsy.json. An append with an empty Uint8Array item dies with a RangeError.

Streamsy.state

declare function state<Self>(): <
  const Name extends string,
  const Template extends string,
  Codecs extends StreamRoute.ParamCodecs,
  const Collections extends StreamAddress.Collections,
>(
  name: Name,
  template: Template,
  options: {
    readonly params: Codecs & StreamRoute.ExactTemplateParams<Template, Codecs>;
    readonly collections: Collections & StreamAddress.ValidCollections<Collections>;
  },
) => Streamsy.StateClass<Self, Name, Template, Codecs, Collections>;

state declares a Durable State stream class. Each key of collections is the type of a collection on the wire. Each entry gives the value schema of the collection and the value field that becomes the key of each change. The items are StreamAddress.CollectionsEvent values, which are changes to any collection and State control messages.

The rules for name, template, and params are the same as for Streamsy.json. Each key must name a string or number field of its own collection, and the compiler reports a wrong key on that collection. To build changes, see DurableState.

Streamsy.custom

declare function custom<Self>(): <const Name extends string, P, A, RD, RE>(
  name: Name,
  options: {
    readonly parse: (id: string) => Option.Option<P>;
    readonly address: (params: P) => StreamAddress.StreamAddress<A, RD, RE>;
  },
) => Streamsy.CustomClass<Self, Name, P, A, RD, RE>;

custom declares a stream class for an id grammar that a template cannot express. parse returns the params of an id that the class owns, or None for any other id. address builds the address for a set of params. parse(address(params).id) must return Some(params), and Streamsy does not check this.

A custom class has an empty template, so a group cannot check it for overlap with other classes. When two classes match an id, the first of them in the group owns the id.

Streamsy.group

declare function group<const Keys extends ReadonlyArray<Streamsy.Any>>(
  ...keys: Keys
): Streamsy.Group<Keys>;

group lists the stream classes that a host stores and serves. The storage Layers, Http.routes, and RemoteStreams.layer take a group. group freezes the group object and its list.

group throws a RangeError in each of these cases:

  • two classes have the same name
  • two classes have the same template
  • two templates can match the same id
  • Streamsy.Wildcard is not the last class

See Groups.

Streamsy.Wildcard

class Wildcard extends Streamsy.custom<Wildcard>()("Wildcard", {
  parse: (id: string) => Option.some({ id }),
  address: ({ id }: { readonly id: string }) => StreamAddress.bytes(id),
}) {}

Wildcard is a custom stream class that owns every id. Its items are bytes with the content type application/octet-stream. To store and serve the ids that no other class owns, put Wildcard last in a group.

Streamsy.Class

type Class<
  Self,
  Name extends string,
  P,
  A,
  RD,
  RE,
  R extends StreamRoute.StreamRoute<P, A, RD, RE>,
> = Context.ServiceClass<Self, `@streamsy/core/Stream/${Name}`, Streamsy.Api<P, A, RD, RE>> &
  R & {
    readonly at: (params: P) => ReturnType<R["address"]> & Streamsy.Address<A, Self, RD, RE>;
  };

Class is the type of every stream class. A stream class has three parts:

PartGives
An Effect serviceyield* Notes returns the Streamsy.Api of the class Notes
The route Rtemplate, match(id), and parse(id)
at(params)The address of one stream, with the stream key of the class

at throws a RangeError when a param does not encode to a valid id segment. Templates and addresses lists the id rules.

RD is the set of services that the codec needs to decode, and RE is the set of services that it needs to encode. Both are never for most schemas.

Streamsy.Api

interface Api<P, A, RD = never, RE = never> {
  readonly owner: object;
  readonly route: StreamRoute.StreamRoute<P, A, RD, RE>;
  readonly wire: Reader<StreamsFault> & Writer<StreamsFault>;
  readonly create: (
    params: P,
    options?: Omit<CreateOptions, "contentType">,
  ) => Effect.Effect<CreateResult, Streams.CreateFailure>;
  readonly append: <const O extends AppendCallOptions = {}>(
    params: P,
    items: Streams.AppendItems<A, O>,
    options?: O,
  ) => Effect.Effect<AppendResult, Streams.AppendFailure<O>, RE>;
  readonly read: (
    params: P,
    options?: ReadOptions,
  ) => Stream.Stream<Streams.Batch<A>, Streams.ReadFailure, RD>;
  readonly fold: <S>(
    params: P,
    initial: Function.LazyArg<S>,
    f: (state: S, item: A) => S,
    options?: ReadOptions,
  ) => Effect.Effect<Streams.Folded<S>, Streams.ReadFailure, RD>;
  readonly collect: (
    params: P,
    options?: ReadOptions,
  ) => Effect.Effect<Streams.Collected<A>, Streams.ReadFailure, RD>;
  readonly decide: <S, B, E, R>(
    params: P,
    spec: Streams.DecideSpec<A, S, B, E, R>,
  ) => Effect.Effect<
    Streams.Decided<B>,
    Streams.ReadFailure | Streams.AppendFailure<{ expectedOffset: string }> | E,
    RD | RE | R
  >;
  readonly follow: (
    params: P,
    options?: Streams.FollowOptions,
  ) => Stream.Stream<Streams.Batch<A>, Streams.FollowFailure, RD>;
  readonly session: (
    params: P,
    options: ReadNextOptions,
  ) => Effect.Effect<ReadNextResult, Exclude<Streams.FollowFailure, DecodeFault>>;
  readonly head: (params: P) => Effect.Effect<HeadResult, Streams.HeadFailure>;
  readonly remove: (params: P) => Effect.Effect<void, Streams.RemoveFailure>;
  readonly fork: (
    params: P,
    source: P,
    options?: Omit<CreateOptions, "contentType" | "forkedFrom">,
  ) => Effect.Effect<CreateResult, Streams.CreateFailure>;
}

Api is the service of a stream class. Each method takes the params of one stream as its first argument, and it works only on the ids of its own class. A call with an id outside the class dies with a RangeError.

MethodDoesGuide
createCreates the stream, with the content type of the addressCreate
forkCreates a child stream from a source of the same classFork
removeDeletes the streamRemove
appendEncodes the items and writes them as one atomic appendAppend
readReads the stored batches after offset, up to the current tailRead
collectReads to the current tail and returns every itemRead
foldReads to the current tail and folds every item into one valueFold
decideFolds the stream, calls your decide function, then appends with expectedOffsetRead, fold, append
followReads to the tail, then waits for new batches until the stream closesFollow
sessionMakes one raw long-poll waitSession
headReturns the tail and the configuration, without messagesRead

append takes a non-empty list of items, unless options has close: true. Its error type lists only the rejections that options make possible. See AppendErrorFor.

fork fails with ForkSourceNotFound when source is outside the class. session, head, and remove do not decode items, so their error types have no DecodeFault.

Projection, Serve, and the storage hosts use owner, route, and wire. An application uses only the methods.

Streamsy.Address

type Address<A, R, RD = never, RE = never> = StreamAddress.StreamAddress<A, RD, RE> & {
  readonly stream: Context.Key<
    R,
    { readonly owner: object; readonly wire: Reader<StreamsFault> & Writer<StreamsFault> }
  >;
};

at(params) returns an Address: a StreamAddress.StreamAddress with the service key of its class as stream. An address names one stream. It holds no stream data, and it is not an Effect Ref. Projection inputs, DurableState.changes, and Serve routes take addresses.

Streamsy.JsonClass

type JsonClass<
  Self,
  Name extends string,
  T extends string,
  C extends StreamRoute.ParamCodecs,
  A,
  I,
  RD,
  RE,
> = Streamsy.Class<
  Self,
  Name,
  StreamRoute.Params<C>,
  A,
  RD,
  RE,
  ReturnType<typeof StreamRoute.json<T, C, A, I, RD, RE>>
>;

Streamsy.json returns a JsonClass. A is the item type, and I is the encoded form of an item.

Streamsy.BytesClass

type BytesClass<
  Self,
  Name extends string,
  T extends string,
  C extends StreamRoute.ParamCodecs,
> = Streamsy.Class<
  Self,
  Name,
  StreamRoute.Params<C>,
  Uint8Array,
  never,
  never,
  ReturnType<typeof StreamRoute.bytes<T, C>>
>;

Streamsy.bytes returns a BytesClass. Its item type is Uint8Array, and its codec needs no services.

Streamsy.StateClass

type StateClass<
  Self,
  Name extends string,
  T extends string,
  C extends StreamRoute.ParamCodecs,
  Collections extends StreamAddress.Collections,
> = Streamsy.Class<
  Self,
  Name,
  StreamRoute.Params<C>,
  StreamAddress.CollectionsEvent<Collections>,
  StreamAddress.CollectionsDecodingServices<Collections>,
  StreamAddress.CollectionsEncodingServices<Collections>,
  ReturnType<typeof StreamRoute.state<T, C, Collections>>
>;

Streamsy.state returns a StateClass. Its decoding and encoding services are the union of the services of every collection schema.

Streamsy.CustomClass

type CustomClass<Self, Name extends string, P, A, RD, RE> = Streamsy.Class<
  Self,
  Name,
  P,
  A,
  RD,
  RE,
  ReturnType<typeof StreamRoute.custom<P, A, RD, RE>>
>;

Streamsy.custom returns a CustomClass. P is the params type that parse returns and address takes.

Streamsy.Group

interface Group<Keys extends ReadonlyArray<Streamsy.Any> = ReadonlyArray<Streamsy.Any>> {
  readonly streams: Keys;
}

Streamsy.group returns a Group. streams holds the classes in the order that you passed them.

Streamsy.Any

interface Any extends Context.Key<
  { readonly [Context.ServiceTypeId]: typeof Context.ServiceTypeId },
  { readonly owner: object; readonly wire: Reader<StreamsFault> & Writer<StreamsFault> }
> {
  readonly template: string;
  readonly match: (id: string) => boolean;
}

Any matches any stream class. It keeps only the parts that a host needs: the service key, template, and match. A group and a storage Layer take a list of Any.

Streamsy.Services

type Services<G extends Streamsy.Group> = G["streams"][number]["Identifier"];

Services<G> is the union of the service types of the classes in the group G.

Streamsy.Host

type Host<G extends Streamsy.Group> = Streamsy.Services<G>;

Host<G> is what a host Layer provides for the group G: the services of its stream classes. MemoryStorage.layer(group) and the other storage Layers provide Host<typeof group>.

On this page