Skip to Content
Streams

Create and fork

Create a stream with an optional expiry, fork a child stream from a source, and remove a stream.

A stream service creates, forks, and removes the streams of its class. Each call takes the params of one stream. For the full signatures, see Streamsy.Api and CreateOptions.

Create

create(params, options?) creates the stream with the content type of its stream class.

import { Effect } from "effect";
import { Notes } from "./streams.ts";

const ensure = Effect.gen(function* () {
  const notes = yield* Notes;
  const result = yield* notes.create({}, { ttlSeconds: 3600 });
  return result.nextOffset;
});
ResultMeaning
Createdcreate made a new stream
ExistsA stream with this id and the same options was already there. create wrote nothing

Both results have nextOffset, contentType, and closed. nextOffset is the tail of the stream. Because an equal call succeeds with Exists, you can call create each time a program starts.

OptionEffect
ttlSecondsThe time to live, in seconds. Each append and each read restarts it
expiresAtA fixed expiry time, as a date string such as 2026-11-01T00:00:00Z
closedtrue creates the stream closed. Every append then fails with StreamClosed
initialDataThe bytes of the first messages, written in the same operation as the create

The options type also has the fork fields forkedFrom, forkOffset, and forkSubOffset. To fork a stream, call fork.

initialData takes raw bytes and skips the codec of the stream class. For a JSON stream, pass the bytes of a JSON array, and each element becomes one message. A JSON value that is not an array becomes one message. Streamsy does not check the elements against schema, so a later read fails with DecodeFault at the first element that does not decode. When a new JSON stream has invalid initialData, create fails with InvalidCreateRequest. For a byte stream, the bytes become one message.

Exists needs the same ttlSeconds, expiresAt, closed, and fork fields as the stored stream. A missing option counts as a different option, so create(params) on a stream that has a TTL fails. create does not compare initialData, and an Exists result writes none of it.

create fails with CreateConflict when the id holds a stream with other options. Its reason is then "config-mismatch". It also fails with CreateConflict when the id holds a removed stream that forks still use, and the reason is then "soft-deleted".

A call can also fail with a fault, such as StorageFault. See Results and errors.

Expiry

A stream with ttlSeconds expires ttlSeconds after its create, its last append, or its last read. head does not restart the TTL. A stream with expiresAt expires at that time, whatever its traffic. A stream with neither option does not expire.

Set ttlSeconds or expiresAt, not both. create and fork fail with InvalidCreateRequest when both are set, including when ttlSeconds is 0. They also fail with InvalidCreateRequest when DateTime.make cannot parse expiresAt. The local protocol checks these options before storage access. The remote Layer checks them before sending a request.

The protocol checks expiry on each call that reaches the id, and deletes an expired stream before the call runs. head, read, append, and remove then fail with StreamNotFound, and create makes a new stream. Calls on an expired stream that forks still use fail with StreamGone, as calls on a removed one do. See Remove.

On Cloudflare, the object alarm deletes expired streams at their deadline. Other hosts have no sweeper. An expired stream stays in storage until a call reaches its id, or until the app runs Protocol.expireDue(). See Run a process server.

A projection that stops reading an input no longer restarts its TTL, so the input can expire. With a checkpoint and no unfinished pinned unit, its next run handles a final gone unit and forgets the binding. Without a checkpoint it fails with history-unavailable; a pinned range that expiry removed fails with range-unreproducible. See Faults.

Fork

A fork is a child stream that starts with a prefix of a source stream. Appends to the child do not change the source. Appends to the source after the fork do not reach the child.

import { Effect } from "effect";
import { ZERO_OFFSET } from "@streamsy/core";
import { Drafts } from "./streams.ts";

const branch = Effect.gen(function* () {
  const drafts = yield* Drafts;
  const copy = yield* drafts.fork({ draftId: "copy" }, { draftId: "main" });
  const empty = yield* drafts.fork(
    { draftId: "empty" },
    { draftId: "main" },
    { forkOffset: ZERO_OFFSET },
  );
  return [copy.nextOffset, empty.nextOffset];
});

fork(params, source, options?) takes the params of the child, then the params of the source. Both are params of the same stream class, and the child has the content type of the source. The child gets every source message up to and including the message at forkOffset. Without forkOffset, the child gets every message of the source up to its tail when the fork runs. With ZERO_OFFSET, the child starts empty.

OptionEffect
forkOffsetThe last source offset that the child gets. The default is the tail of the source
forkSubOffsetThe child also gets the start of the next source message. For a JSON stream it counts messages, otherwise bytes. A value above 0 needs forkOffset
ttlSecondsAs for create. A fork that sets neither ttlSeconds nor expiresAt takes the expiry option of its source
expiresAtAs for create
closedAs for create
initialDataMessages written after the prefix, as for create

fork returns a CreateResult. A second fork with the same child, source, and options succeeds with Exists.

StorageFork
MemoryStorage.layerThe child shares the prefix of the source
MemoryStorage.layer with constrained: trueThe child gets a copy of the prefix
@streamsy/storage (SQLite, Postgres, MySQL, and Durable Object storage)The child shares the prefix of the source
A storage adapter with fork: "none"fork fails with NotSupported

A fork never crosses two storage Layers. On Cloudflare, each Durable Object has its own storage, so the source and the child must be on the same object. A fork whose source is on another object fails with ForkSourceNotFound.

ErrorWhen
InvalidCreateRequestBoth expiry options are set, or expiresAt cannot be parsed by DateTime.make
ForkSourceNotFoundThe source does not exist or has expired
InvalidForkRequestforkOffset is not a valid offset or is past the tail of the source, or forkSubOffset is past the next source message, or JSON initialData for a new fork is invalid
CreateConflictThe child id holds a stream with other options, or the source is a removed stream that forks still use (reason is "fork-source-soft-deleted")
NotSupportedThe storage has no fork support

See Fork in the Durable Streams docs.

Remove

remove(params) deletes the stream and its messages. Later calls on the id fail with StreamNotFound, and create makes a new, empty stream.

A stream that forks still use stays in storage after remove, so the forks keep their prefix. Calls on its id fail with StreamGone, and create fails with CreateConflict with reason "soft-deleted". Storage deletes the stream when its last fork is removed. With constrained: true, each fork has a copy of its prefix, so remove deletes the source at once.

ErrorWhen
StreamNotFoundNo stream has the id
StreamGoneThe stream was removed, and forks still use it
StreamBusyOver the remote Layer only. The server answered 503. Try again later

Do not reuse a stream id. Readers, projections, and wake streams keep an offset as a cursor into one stream. After remove or expiry, create on the same id makes a new stream that starts again at ZERO_OFFSET, and Streamsy does not detect the reuse. A reader with a saved offset skips the messages of the new stream up to that offset. A projection stops with history-unavailable while the tail of the new stream is below its saved offset. Give each new stream a new id.

Over HTTP

Http.routes serves create, fork, and remove for a stream only when its access is "full". With "read" or "append", the server answers 405 with an allow header. See Expose the stream protocol.

RequestOperationSuccess
PUTCreate201 with location, or 200 when the stream exists with the same options
PUT with Stream-Forked-FromFork201 with location, or 200 when the fork exists with the same options
DELETERemove204

A PUT reads these request headers. The body, when it is not empty, is initialData.

HeaderValue
Content-TypeThe content type of the stream. The default is application/octet-stream, or the content type of the source for a fork
Stream-TTLttlSeconds, as a whole number
Stream-Expires-AtexpiresAt
Stream-Closedtrue creates the stream closed
Stream-Forked-FromThe source of a fork, as a path under the prefix or as a stream id
Stream-Fork-OffsetforkOffset. Needs Stream-Forked-From
Stream-Fork-Sub-OffsetforkSubOffset. Needs Stream-Forked-From

A successful PUT answers with content-type and stream-next-offset, and with stream-closed: true when the stream is closed.

Http.routes does not apply the content type of the stream class. A PUT without Content-Type creates an application/octet-stream stream, and a typed create on that id then fails with CreateConflict. See What the server checks.

StatusWhen
400A header is invalid, a PUT has both Stream-TTL and Stream-Expires-At, Stream-Expires-At cannot be parsed, a JSON body does not parse, or the fork is invalid (InvalidForkRequest). A fork that the storage does not support also answers 400, with a stream-not-supported header
404No stream in the group matches the id, a DELETE finds no stream, or the fork source does not exist
409CreateConflict
410A DELETE on a removed stream that forks still use

The remote Layer maps a 400 response to InvalidCreateRequest for a plain create and InvalidForkRequest for a fork. A 400 response with stream-not-supported becomes NotSupported instead. See Protocol errors.

On this page