Skip to Content
Reference@streamsy/core

Wake streams

Reference for the wake stream members of the Streamsy namespace in @streamsy/core. A wake stream tells a client which source streams moved.

import { Streamsy, StreamAddress, type FollowFailure, type StreamId } from "@streamsy/core";
import type { StreamRoute, Streams } from "@streamsy/core/protocol";
import type { Context, Data, Schema, Stream } from "effect";

A wake stream is a Durable State stream with one row for each source stream in a family. Each row holds the tail of its source after the last write, so a client follows one wake stream and reads only the sources that moved past its saved cursors. These members belong to the Streamsy namespace.

A wake declaration names no sources. Projection.wake from @streamsy/projection writes the rows: it links one source stream to one wake stream. For a host that runs Projection.wake, see Projections on Cloudflare.

Functions

Streamsy.wake

declare function wake<Self>(): <
  const Name extends string,
  const Template extends string,
  C extends StreamRoute.ParamCodecs,
>(
  name: Name,
  template: Template,
  options: { readonly params: C & StreamRoute.ExactTemplateParams<Template, C> },
) => Streamsy.StateClass<Self, Name, Template, C, Streamsy.WakeCollections>;

wake declares a wake stream class. It calls Streamsy.state with wakeCollections as the collections, so the class has the normal State service, including create, append, remove, read, follow, and collect.

import { Schema } from "effect";
import { Streamsy } from "@streamsy/core";

class UserWake extends Streamsy.wake<UserWake>()("App/UserWake", "users/:user/wake", {
  params: { user: Schema.String },
}) {}

The rules for name, template, and params are the same as for Streamsy.json. A wake stream on another storage owner must exist before a projection appends to it, so that owner creates the wake stream when it starts.

wakeActions compares each row offset with a saved cursor, so it assumes that the offsets of a source stream only grow. Do not create a stream again under the id of a removed or expired stream. The new stream starts at a lower offset, and a client with a saved cursor for the old stream skips its items.

Streamsy.followWake

declare function followWake<Self, P>(
  declaration: Context.Key<Self, Streamsy.Api<P, Streamsy.WakeEvent>>,
  params: P,
  options: Streamsy.FollowWakeOptions,
): Stream.Stream<Streamsy.WakeBatch, FollowFailure, Self>;

followWake follows the wake stream params of declaration from options.from, and emits one WakeBatch for each batch that it reads. It calls wakeActions on each batch and carries an open snapshot from one batch to the next. It reads no source stream.

followWake reads options.cursors again for each batch, so you can update the map as you read the sources. Save each source cursor first and the wake cursor offset last. When the process stops between the two, the next run reads the same rows again, and wakeActions skips each source whose saved cursor already reaches the row offset.

import { Streamsy, type StreamId } from "@streamsy/core";
import { UserWake } from "./wake.ts";

const cursors = new Map<StreamId, string>();
const batches = Streamsy.followWake(UserWake, { user: "ann" }, { from: "-1", cursors });

Streamsy.awaitWake

declare function awaitWake<Self, P>(
  declaration: Context.Key<Self, Streamsy.Api<P, Streamsy.WakeEvent>>,
  params: P,
  target: { readonly stream: StreamId; readonly offset: string },
  options?: Streamsy.AwaitWakeOptions,
): Effect.Effect<void, FollowFailure, Self>;

Wait for the source row to reach the requested offset under core compare. The helper catches up before checking, respecting resets, removals and snapshot controls across history pages. It uses the wake service over local or remote storage. AwaitWakeOptions is Omit<Streamsy.FollowWakeOptions, "cursors">: pass from to resume after a saved wake cursor, plus any Streams.FollowOptions except offset. Without from, each call reads the wake stream from the start. A saved cursor skips earlier rows, so the target must be observed after that cursor. Callers choose their own Effect.timeout; a closed wake stream without a matching row leaves the wait pending. See Accepted stream.

Streamsy.wakeActions

declare function wakeActions(
  events: ReadonlyArray<Streamsy.WakeEvent>,
  options: Streamsy.WakeActionsOptions,
): Streamsy.WakeActions;

wakeActions returns the actions for a batch of wake events. It reads no stream and changes none of its inputs. A client with its own HTTP reader decodes each page with WakeMessage and calls wakeActions directly. Pass the returned snapshot to the call for the next page.

wakeActions applies these rules, in event order:

  • It collapses the rows of one stream in the batch to one item, at the position of the last row for that stream. The one exception is a delete and then an upsert, below.
  • An upsert gives Dirty when the row offset sorts after the saved cursor of the stream, or when the row has closed. The saved cursor is the entry in cursors, or -1 when the map has none.
  • A delete gives Removed. A delete and then an upsert of the same stream give Removed, then Dirty from -1.
  • A reset control message drops the items before it in the batch, closes an open snapshot, and gives Reset. Each later upsert in the batch gives Dirty from -1.
  • A snapshot-start control message opens a snapshot. The snapshot-end control message that closes it gives the items of the rows before it, then Snapshot. Snapshot holds each stream that an upsert named since snapshot-start and that no later delete removed.

A snapshot-end with no open snapshot gives no item.

Rows and messages

Streamsy.WakeRow

type WakeRow = {
  readonly stream: StreamId;
  readonly offset: Offset;
  readonly closed?: true;
};

WakeRow is a row of a wake stream: the latest known state of a source stream. Streamsy.WakeRow is also the Schema of a row.

FieldMeaning
streamThe id of the source stream. It is also the State key of the row
offsetA source position the writer vouches for: written in a source wake stream, accepted in an accepted stream
closedtrue when the source stream is closed. A row for an open stream leaves it out

A row lags its source by one projection pass, because only the projection writes the row. A write to the source alone writes no row.

Streamsy.wakeCollections

declare const wakeCollections: {
  readonly stream: { readonly schema: typeof Streamsy.WakeRow; readonly key: "stream" };
};

wakeCollections holds the only collection of every wake stream. Its State type is stream, and its key is the stream field of the row. A class from Streamsy.wake has wakeCollections as its collections.

Streamsy.WakeCollections

type WakeCollections = typeof Streamsy.wakeCollections;

WakeCollections is the type of wakeCollections.

Streamsy.WakeEvent

type WakeEvent = StreamAddress.CollectionsEvent<Streamsy.WakeCollections>;

WakeEvent is an item of a wake stream: a change to a WakeRow, or a State control message. See StreamAddress.CollectionsEvent.

Streamsy.WakeMessage

declare const WakeMessage: Schema.Codec<
  Streamsy.WakeEvent,
  StreamAddress.CollectionsEncodedEvent<Streamsy.WakeCollections>
>;

WakeMessage is the codec of a wake stream message: a stream change or a State control message. A client that reads raw JSON pages decodes each message with it, in place of a typed service.

import { Schema } from "effect";
import { Streamsy } from "@streamsy/core";

const decodePage = Schema.decodeUnknownEffect(Schema.Array(Streamsy.WakeMessage));

Actions

Streamsy.WakeItem

type WakeItem = Data.TaggedEnum<{
  Dirty: {
    readonly stream: StreamId;
    readonly from: string;
    readonly offset: string;
    readonly closed: boolean;
  };
  Removed: { readonly stream: StreamId };
  Reset: {};
  Snapshot: { readonly streams: ReadonlySet<StreamId> };
}>;

declare const WakeItem: Data.TaggedEnum.Constructor<Streamsy.WakeItem>;

WakeItem is an action for the consumer of a wake stream. The value Streamsy.WakeItem holds a constructor for each tag, and $is and $match.

TagAction
DirtyRead stream from from up to at least offset, then save its cursor
RemovedDrop the cursor and the local data of stream
ResetDrop every cursor. The items after this one start each stream from -1
SnapshotA complete snapshot ended. Drop each stream that streams does not hold

closed on Dirty is true when the row says that the source stream is closed.

Streamsy.WakeBatch

interface WakeBatch {
  readonly items: ReadonlyArray<Streamsy.WakeItem>;
  readonly offset: string;
  readonly upToDate: boolean;
}

WakeBatch holds the actions of a batch of the wake stream. followWake emits one for each batch that it reads. offset is the nextOffset of that batch. Save it as the wake cursor after you handle every item. upToDate is true when the batch reached the tail of the wake stream.

Streamsy.FollowWakeOptions

interface FollowWakeOptions extends Omit<Streams.FollowOptions, "offset"> {
  readonly from?: string;
  readonly cursors: ReadonlyMap<StreamId, string>;
}

FollowWakeOptions holds the options of followWake. It takes the follow options of a stream, such as retry, with from in place of offset.

FieldMeaning
fromThe saved wake cursor. The default is -1, which reads the wake stream from the start
cursorsThe saved source cursors, by stream id. followWake reads the map again for each batch

Streamsy.WakeActionsOptions

interface WakeActionsOptions {
  readonly cursors: ReadonlyMap<StreamId, string>;
  readonly snapshot?: ReadonlySet<StreamId> | undefined;
}

WakeActionsOptions holds the options of wakeActions.

FieldMeaning
cursorsThe saved source cursors, by stream id. A stream with no entry starts from -1
snapshotThe snapshot that the call for the previous page returned. Leave it out for the first page

Streamsy.WakeActions

interface WakeActions {
  readonly items: ReadonlyArray<Streamsy.WakeItem>;
  readonly snapshot: ReadonlySet<StreamId> | undefined;
}

WakeActions is the result of wakeActions. items holds the actions of the batch, in order. snapshot holds the streams of a snapshot that is still open at the end of the batch, or undefined when no snapshot is open. Pass snapshot to the call for the next batch.

On this page