Skip to Content
Streams

Durable State streams

Declare a stream of keyed rows, write change events and control messages, and fold the rows on a read.

A Durable State stream holds keyed rows in collections. A writer appends change events, and each event sets or deletes one row. A reader folds the events into the current rows. The format is the Durable State protocol from the Durable Streams docs, so a browser can read the stream with @durable-streams/state.

Declare

Streamsy.state<Self>()(name, template, { params, collections })

import { Schema } from "effect";
import { Streamsy } from "@streamsy/core";

const Project = Schema.Struct({ id: Schema.String, name: Schema.String });
const Issue = Schema.Struct({ id: Schema.String, projectId: Schema.String, title: Schema.String });

export class Workspace extends Streamsy.state<Workspace>()(
  "Tracker/Workspace",
  "workspaces/:workspaceId",
  {
    params: { workspaceId: Schema.String },
    collections: {
      project: { schema: Project, key: "id" },
      issue: { schema: Issue, key: "id" },
    },
  },
) {}

Each key of collections is a collection type. Its entry gives the value schema and the key field of that collection. The compiler checks that the key field is a string or number field of the schema. The service name, template, and params work as for every stream class. See Declare streams. For the full signature, see Streamsy.state.

Items

The item type is a union of change events and control messages.

ItemFields
Upserttype, key, value, and headers with operation: "upsert"
Deletetype, key, an optional old_value, and headers with operation: "delete"
Controlheaders with control set to snapshot-start, snapshot-end, or reset. A control has no type and no key

type is the collection type and key is the row key as a string. The headers of a change can also hold offset, txid, timestamp, and from. The headers of a control can hold offset.

Narrow an item with StreamAddress.isChange, StreamAddress.isUpsert, or StreamAddress.isControl before you read its type or key. A read decodes the insert and update operations of other producers as upsert. See Changes and controls for the types.

Write changes

DurableState.changes makes the change events for an address of a Streamsy.state stream.

import { Effect } from "effect";
import { DurableState } from "@streamsy/core";
import { Workspace } from "./streams.ts";

const createIssue = (offset: string, issue: { id: string; projectId: string; title: string }) =>
  Effect.gen(function* () {
    const workspace = yield* Workspace;
    const params = { workspaceId: "main" };
    const facts = DurableState.changes(Workspace.at(params), { offset }, [
      DurableState.upsert("issue", issue),
    ]);
    yield* workspace.append(params, facts);
  });
FieldValue
keyThe collection key field of an upserted value, or the key given to remove
offsetThe source offset you give
txid${offset}:${index}, unless the change sets it

DurableState.remove("issue", id) deletes by key. Add { oldValue } to also write old_value. If you give no offset, the events have no offset and no default txid. An upsert or delete can also set timestamp or from. A change to an undeclared collection, or a value that does not match its schema, does not compile. See DurableState for the full signatures.

Control messages

DurableState.changes also takes the State control messages. Make them with StreamAddress.reset(), StreamAddress.snapshotStart() and StreamAddress.snapshotEnd(). Each takes an optional { offset }.

import { Effect } from "effect";
import { DurableState, StreamAddress } from "@streamsy/core";
import { Workspace } from "./streams.ts";

const replaceIssues = (issues: ReadonlyArray<{ id: string; projectId: string; title: string }>) =>
  Effect.gen(function* () {
    const workspace = yield* Workspace;
    const params = { workspaceId: "main" };
    const facts = DurableState.changes(Workspace.at(params), {}, [
      StreamAddress.reset(),
      ...issues.map((issue) => DurableState.upsert("issue", issue)),
    ]);
    yield* workspace.append(params, facts);
  });

A control passes through changes unchanged. With an offset, a control that has no offset of its own gets that offset. Controls do not take a txid, but they count in the index of the default txid of the changes after them.

Conditional writes

A State stream takes expectedOffset and decide like any other stream. See Conditional writes. Pass the offset to DurableState.changes as well, so each change records the offset that the append checks. The issue tracker does this in IssueTracker.mutate.

Read and fold rows

A Durable State read gives the change events and the control messages that the stream holds, in order. To fold the rows yourself, clear them at a reset and apply the items after it.

DurableState.materialize folds items into rows with the rules of the protocol. Start it from DurableState.materialized, which gives one empty map for each collection.

import { Effect } from "effect";
import { DurableState } from "@streamsy/core";
import { Workspace } from "./streams.ts";

const issues = Effect.gen(function* () {
  const workspace = yield* Workspace;
  const { items } = yield* workspace.collect({ workspaceId: "main" });
  return DurableState.materialize(DurableState.materialized(Workspace), items).rows.issue;
});

rows.issue is a ReadonlyMap from key to value. A snapshot-end removes each row that no upsert named since the snapshot-start before it. See DurableState.materialize for every rule.

Serve and project

On this page