watershed Collaborative data structures for Gleam

← watershed · Models / SharedTree

Two places to put
the guarantee.

Fluid Framework’s SharedTree and watershed’s typed layer want the same thing: declare a document’s shape once, get static types, get roots that are guaranteed to be there, get told when something changes. They disagree about where to put the guarantee, and almost everything else follows from that choice.

The one difference the rest follows from

SharedTree makes the schema the document. The SchemaFactory id you pick is written into the container as the stored schema, and from then on every client’s SharedTree enforces it. A client whose view schema disagrees cannot silently write the wrong shape. Because there is one tree under one rebasable changeset algebra, transactions, undo/redo, node identity, cross-parent moves, and a real schema-upgrade path all fall out of the same machinery.

watershed makes the schema a decode boundary that exists only in your build. The tag scoping a set of fields is a phantom type, erased at compile time. Nothing about it reaches the wire, and any Fluid Framework-compatible sequencer (floodgate included) stays content-agnostic by design. In exchange the document is not one structure but a graph of independently converging channels, so the merge policy is named per slot rather than once for everything. Reads hand back a Result, because a peer really can write anything. What you do not get is atomicity, undo, or a schema upgrade. Those are listed further down, not buried.

Both columns below model the same board. The Gleam is adapted from examples that compile today: retro_board_lustre, scoreboard_cli, sudoku_lustre. The TypeScript uses only APIs the Fluid Framework documentation publishes.

1 · Declaring the shape

Both declare once and derive the types. The difference is what the declaration is: a SchemaFactory id becomes part of the document, where pub type Board is a phantom tag that exists only to stop you reading a field against the wrong map. Note the counter too: SharedTree’s wipBreaches is a number in the tree; watershed’s is a handle to a whole SharedCounter, with commutative increments of its own. Foundations · schemas and fields covers that phantom tag, and the three kinds of field, in full.

Fluid SharedTree · TypeScript
import { SchemaFactory } from "fluid-framework";

// This id is written into the document. It becomes the *stored* schema,
// and from here on every client's SharedTree enforces it.
const sf = new SchemaFactory("com.example.sprintboard");

class Card extends sf.object("Card", {
  title: sf.string,
  column: sf.string,
  owner: sf.optional(sf.string),
}) {}

class Cards extends sf.array("Cards", Card) {}

class Board extends sf.object("Board", {
  title: sf.string,
  cards: Cards,
  wipBreaches: sf.number,
}) {}
watershed · src/document_schema.gleam
import gleam/dynamic/decode
import gleam/json
import watershed/schema.{
  type ChannelField, type CounterChannel, type Field, type MapChannel,
}

// A phantom tag. It is erased at compile time — nothing about `Board`
// reaches the document, and no peer has to agree it exists.
pub type Board

pub fn title() -> Field(Board, String) {
  schema.field("title", json.string, decode.string)
}

pub fn cards() -> ChannelField(Board, MapChannel) {
  schema.channel_field("cards")
}

pub fn wip_breaches() -> ChannelField(Board, CounterChannel) {
  schema.channel_field("wip_breaches")
}

2 · Establishing one root

initialize is a once-only act against an empty document; it writes the stored schema and every later client reads it back. ensure_* is the opposite shape: idempotent, and run by every client on every boot. Two clients opening a fresh document both seed a candidate channel; each one waits for its own write to sync, then adopts whichever handle the field shows at that moment — a concurrent write from the other client can still land after and replace the field. The field itself converges on one handle without a leader; the handle each call returns just isn't promised to be that one. Foundations · starting a document walks through that whole bootstrap in order.

Fluid SharedTree · TypeScript
import { TreeViewConfiguration } from "fluid-framework";

const config = new TreeViewConfiguration({ schema: Board });
const view = tree.viewWith(config);

// Once, against an empty document. After this the stored schema is
// authoritative and every later client just reads it.
view.initialize(
  new Board({ title: "Sprint board", cards: [], wipBreaches: 0 }),
);

const board = view.root; // typed Board, guaranteed to exist
watershed · Lustre effects
fn bootstrap(document: Document(document_schema.Board)) -> Effect(Msg) {
  let root = watershed.root_typed(document)
  effect.batch([
    // Idempotent, and run by every client on every boot. Each one
    // seeds a candidate and adopts the handle visible after its own
    // sync — a later concurrent write can still replace the field,
    // so the channel handed back here isn't promised to be final.
    watershed_lustre.ensure_field(
      root,
      document_schema.title(),
      "Sprint board",
    ),
    watershed_lustre.ensure_map(
      document,
      root,
      document_schema.cards(),
      EnsuredCards,
    ),
    watershed_lustre.ensure_counter(
      document,
      root,
      document_schema.wip_breaches(),
      EnsuredBreaches,
    ),
  ])
}

The losers stay attached. A candidate channel that did not win the race is still attached to the document, just unreferenced. Orphan collection is out of scope; that’s the cost of not having a leader elect the root.

3 · Reading and writing a field

This is the whole argument in eight lines. board.title is a string, and under a stored schema it cannot be anything else, so the read needs no error arm. watershed cannot promise that, so it does not pretend to: get_field returns Result(Option(a), FieldError), and you confront the “a peer disagrees about this type” case once, at the read, instead of discovering it later as corrupted state.

Fluid SharedTree · TypeScript
// Writing is checked by the compiler and again by SharedTree.
board.title = "Q3 sprint board";

// Reading is total. `title` is a string, and no client running this
// schema can make it anything else.
renderHeader(board.title);
watershed · Gleam
// Writing is checked at the field — only a String is accepted here.
watershed.set_field(root, document_schema.title(), "Q3 sprint board")

// Reading decodes at the boundary. A peer running an older build — or
// a stale summary — could have written anything here, so the result is
// a `Result`, never a raw value.
case watershed.get_field(root, document_schema.title()) {
  Ok(Some(title)) -> render_header(title)
  Ok(None) -> render_header("Untitled board")  // key absent
  Error(_) -> render_header("Untitled board")  // a peer wrote a number
}

Whether that Error arm is a tax or a feature depends on what you can assume about the clients. If every writer runs your build against an enforced stored schema, it is noise. If an old tab, a stale summary, or a second application can reach the same document, it is the one place a multiplayer app gets to decide what to do about disagreement. The build guide works through it in context.

4 · A record spread across keys

SharedTree stores the node, so the class declaration is the only declaration there is. watershed stores JSON per key, which normally means writing a decoder and an encoder and keeping them in sync by hand. The record1record9 builders derive both from one prop list instead, and sealed_known seals the schema without you repeating the key names a third time.

Fluid SharedTree · TypeScript
// The shape was declared once, on the class. There is no second
// encoder to keep in sync, because the tree stores the node itself.
card.title = "Ship the gauge rebuild";
card.column = "doing";
card.owner = undefined; // clears the optional property
watershed · Gleam
/// `record3` derives the decoder AND the per-key encoder from one
/// prop list, so the two can never drift. `sealed_known` seals the
/// schema to exactly these keys without a hand-repeated list, and
/// `versioned` stamps a version and rejects any stored version that differs.
fn card_schema() -> Result(schema.Schema(Card, CardState), Nil) {
  schema.record3(
    CardState,
    schema.prop(card_title(), fn(c: CardState) { c.title }),
    schema.prop(card_column(), fn(c: CardState) { c.column }),
    schema.optional_prop(card_owner(), fn(c: CardState) { c.owner }),
  )
  |> schema.versioned(1)
  |> schema.sealed_known
}

// `write` emits one op per key, never a blob — so a peer editing
// `owner` at the same time keeps their edit. An optional prop that is
// `None` deletes its key rather than skipping it.
use card_schema <- result.try(card_schema())
watershed.write(
  card,
  card_schema,
  CardState(title: "Ship it", column: "doing", owner: None),
)

Per-key ops cut both ways. Writing a whole record as one op per key is why a peer editing owner at the same moment keeps their edit: the record view is never a clobbering blob. It is also why a peer can read the record mid-write and see three of four keys updated. Tree.runTransaction would prevent that; watershed has no answer for it yet. See below.

5 · One tree, or a graph of channels

Here the two models stop resembling each other. Everything in a SharedTree is in the tree, under one changeset algebra. That is what makes a reorder across two arrays a coherent operation, and what makes the merge behaviour uniform enough that you never have to think about it.

A watershed document is not one structure. Each slot on the root holds a handle to a channel that converges on its own terms, and the field declaration names which terms. The Sudoku example is the clearest case: four slots, four different answers to “what happens when two clients touch this at once”. Foundations · documents and handles lays out that layout on its own terms, without the SharedTree comparison.

Fluid SharedTree · TypeScript
// Everything is in one tree under one changeset algebra, so inserts,
// removals, and reorders are all edits to the same document — and all
// merge by the same rules.
board.cards.insertAtEnd(new Card({ title: "Ship it", column: "todo" }));
board.cards.moveRangeToIndex(4, 0, 3);
board.cards.removeAt(2);
watershed · sudoku_lustre/document_schema.gleam
/// One root, four merge policies — named per slot, at declaration time.
pub type SudokuDocument

pub fn cells() -> ChannelField(SudokuDocument, MapChannel) {
  schema.channel_field("cells") // last write wins, per key
}

pub fn notes() -> ChannelField(SudokuDocument, OrSetChannel) {
  schema.channel_field("notes") // add wins over a concurrent remove
}

pub fn givens() -> ChannelField(SudokuDocument, ClaimsChannel) {
  schema.channel_field("givens") // the first writer owns the slot
}

pub fn mistakes() -> ChannelField(SudokuDocument, CounterChannel) {
  schema.channel_field("mistakes") // commutative increments
}

So watershed makes you choose, and in exchange lets you choose right: pencil marks want an OR-set where a concurrent add beats a remove, the puzzle’s givens want first-writer-wins claims, the mistake tally wants a counter that does not double-count a redelivered op. SharedTree would give all four the same treatment. The DDS · CRDT · OT comparison lays out what each of those policies costs.

Nesting is indirection, not containment. A nested typed map, schema.child_field("players"), stores a handle, and the child is a peer channel, not a subtree of its parent. There is no cross-parent move, no operation that spans two channels, and a root made of child fields cannot be sealed by a record schema at all. See Foundations · documents and handles for what that costs you in practice.

6 · Being told what changed

treeChanged is the thing a tree buys you that a graph cannot: a rollup over an entire subtree, for free, because the subtree is a real thing with a boundary. watershed has no rollup above a single channel: you subscribe per channel, and per field.

What a field subscription gives back in exchange is decoded on both sides. A FieldChange carries the previous value and the new one, already run through the field’s decoder, plus whether this client made it. A peer’s type-confused write arrives as an Error in the event rather than as state that looks fine until it does not.

Fluid SharedTree · TypeScript
// Per node, plus a subtree rollup for free: `treeChanged` fires for a
// change anywhere below the node, because there is a tree to roll up.
const stopCards = Tree.on(board.cards, "nodeChanged", renderCards);
const stopAll = Tree.on(board, "treeChanged", renderEverything);

// Editing the tree from inside a change callback throws a UsageError.
watershed · Gleam
// Per field. The change carries the decoded previous value AND the
// new one, plus whether this client made it — and a peer's
// type-confused write surfaces here as `Error(Invalid(_))`.
watershed.subscribe_field(root, document_schema.title(), fn(change) {
  case change {
    FieldChange(value: Ok(Some(title)), local: False, ..) ->
    render_header(title)
    FieldChange(value: Error(_), ..) -> keep_previous_header()
    _ -> Nil
  }
})
watershed · sudoku_lustre, narrowed per-kind subscriptions
// Or subscribe per channel. Each handler sees only its own channel's
// event type — a counter subscriber never pattern-matches a map event.
effect.batch([
  watershed_lustre.subscribe(shared.cells, fn(_event) { SharedChanged }),
  watershed_lustre.subscribe_or_set(shared.notes, fn(_event) { SharedChanged }),
  watershed_lustre.subscribe_claims(shared.givens, fn(_event) { SharedChanged }),
  watershed_lustre.subscribe_counter(shared.mistakes, fn(_event) {
    SharedChanged
  }),
])

What SharedTree has that watershed does not

All three of these are consequences of the same choice. A rebasable changeset over one enforced tree is what lets you group edits, revert them, and move a document from one stored schema to the next. watershed has ops over independent channels and a client-side decoder, so it has none of them.

Fluid SharedTree · TypeScript
// Atomicity — every edit in the callback lands, or none of them does.
Tree.runTransaction(board, () => {
  card.column = "doing";
  card.owner = "ada";
  if (overWipLimit(board)) return Tree.runTransaction.rollback;
});

// Undo — each local commit offers a revertible.
view.events.on("commitApplied", (data, getRevertible) => {
  if (getRevertible !== undefined) undoStack.push(getRevertible());
});

// Schema upgrade — an old document can be moved onto the new stored schema.
if (!view.compatibility.canView && view.compatibility.canUpgrade) {
  view.upgradeSchema();
}
  • Transactions. watershed writes per-key ops, so peers can observe a partial record. Fixing it means a batched multi-set map op: a wire-format change, deliberately deferred until a real consumer hurts.
  • Undo and redo. Not offered. The kernels do have a rollback, but that is nack-and-resubmit machinery for unacknowledged local ops, not a user-facing stack.
  • Schema upgrade. schema.versioned(n) stamps a version and fails closed on mismatch, which is a dead end rather than a path. Entries-level migrations are designed and unshipped.
  • Enforcement, node identity, cross-parent moves, branching. Absent by design. Typing stays a client decode boundary; the server stays content-agnostic.

The reason is worth stating as a number rather than an adjective. Fluid Framework’s tree package (stored-versus-view schema system, forest, compositional changeset rebasing, undo/redo, ID compression) is 77,804 lines — a multi-year project to reproduce in any language, and one watershed would take on only for a compelling product need. watershed’s entire schema vocabulary is one roughly 700-line module: src/watershed/schema.gleam. That is the trade.

Side by side

The same axes, two answers. Read the bottom four rows together with the top one: they are the same decision, seen from either end.

Property SharedTree watershed
Schema location in the document, as a stored schema written on first initialize in your build only, as a phantom tag and a set of field descriptors erased at runtime
A peer writes the wrong type cannot happen under the stored schema; SharedTree rejects the write on that client reaches the document, and surfaces as Error(Invalid(_)) on your read
Shape of the data one tree, one changeset algebra over the whole document a graph of independently addressed channels, joined by handles stored as map values
Merge policy one, applied to everything in the tree one per slot: LWW map, add-wins OR-set, first-writer-wins claims, commutative counter, OT rich text
Declare once SchemaFactory classes; static types derived Field / ChannelField / recordN; encoder and decoder derived from one prop list
Guaranteed roots viewWith + initialize, once, against an empty document ensure_* per slot, idempotent, run by every client on every boot
Invalidation nodeChanged plus a treeChanged subtree rollup subscribe_field (decoded previous and new) and narrowed per-kind channel events; no rollup above a channel
Atomic multi-field write Tree.runTransaction, with rollback and preconditions none: per-key ops, so peers can observe a partial record
Undo / redo commitApplied → Revertible.revert() none offered
Schema upgrade canView / canUpgrade / upgradeSchema, plus staged rollout versioned(n) fails closed with SchemaMismatch; no migration path
Moves moveRangeToIndex, including between arrays in the same tree convergent move inside one SharedSequence; no cross-parent move
Runs on TypeScript and JavaScript one Gleam core, on the BEAM and in the browser
Implementation size 77,804 lines in the Fluid tree package about 700 lines in watershed/schema.gleam