watershed — connect the board

Connect to a live document

A watershed document is a collection of data stored in collaborative structures. Watershed defines how those structures sync; your application decides what their data means. In this tutorial, each document is one retro board.

A tenant id and a document id address the document. The tenant id groups documents; the document id picks one board in that group. Any client that opens the same pair of names joins the same live board — open the same link in two tabs and you’re looking at the same board in both.

Connecting does two things. First it catches you up: the server replays the document’s whole history of edits, so this board opens at its current state instead of an empty one. Then it keeps the line open, streaming every later edit — yours and everyone else’s — in the single order the server assigns.

src/retro_tutorial_lustre.gleam

/// These dev constants match `just integration-up`.
/// Change them when you point the example at another server.
const socket_url = "ws://localhost:4000/socket/websocket?vsn=2.0.0"

const tenant = "dev-tenant"

const tenant_secret = "levee-dev-secret-change-in-production"

pub fn main() -> Nil {
  let app = lustre.application(init, update, view)
  let document = browser.document_on_navigate("retro-tutorial")
  let assert Ok(_) = lustre.start(app, "#app", document)
  Nil
}

document_on_navigate reads ?document= from the URL, or mints a name and puts it there — so two tabs share a board by sharing the link

Connecting is a declared effect, not a callback

The browser side of watershed talks in callbacks: on_ready, subscription handlers, presence events. If you wired those into Lustre by hand, each one would need to fire off a message on its own — and firing a message from inside update, while update is still assembling the model you’re about to return, quietly overwrites that half-built model with a stale one. watershed_lustre does that wiring once, correctly, so your init just returns the connection as one more effect — the same way it returns any other effect in Lustre.

fn init(document: String) -> #(Model, Effect(Msg)) {
  let user_id = "web-" <> int.to_string(1000 + int.random(9000))
  let model =
    Model(
      status: Connecting,
      document: None,
      shared: None,
      pending: PendingShared(None, None),
      user_id: user_id,
      color: presence.color_for(user_id),
      focus: None,
      presence: None,
      peers: [],
      board: board.empty("Sprint retro"),
      drafts: dict.new(),
      last_error: None,
    )

  #(
    model,
    watershed_lustre.connect_dev(
      url: socket_url,
      tenant: tenant,
      secret: tenant_secret,
      document: document,
      user_id: user_id,
      got_document: GotDocument,
      connected: Connected,
    ),
  )
}

Development and production put token signing on opposite sides of the browser boundary. connect_dev keeps this tutorial self-contained by signing an HS256 token in the browser with the tenant secret expected by just integration-up. A production browser cannot hold that secret: anyone who extracts it could mint tokens for the tenant. Your backend must authenticate the user, sign the token, and send only that token to the browser, which passes it to watershed_lustre.connect. The two functions make that credential boundary explicit; after token creation, both use the same connection path.

The handle and the handshake are two different events

Notice that connect_dev fires two separate messages. got_document arrives the moment you have something to hold onto — a reference to the document, before it’s caught up. connected arrives later, once the server has finished replaying history and you’re actually in sync. Treat those as the same moment and you get a subtle bug on first load: you start building on a document that looks ready but hasn’t caught up yet, and wonder why the board flickers or shows nothing.

So the tutorial app keeps both messages separate, and only starts building the board once whichever one arrives second has shown up. That is the whole reason for the double case below.

    GotDocument(document) -> {
      let model = Model(..model, document: Some(document))
      let presence_start = presence_effect(model, document)
      case model.status, model.shared {
        Ready, None -> #(
          model,
          effect.batch([bootstrap_effect(document), presence_start]),
        )
        _, _ -> #(model, presence_start)
      }
    }

    Connected(Ok(_)) -> {
      let model = Model(..model, status: Ready)
      case model.document, model.shared {
        Some(document), None -> #(model, bootstrap_effect(document))
        _, _ -> #(model, effect.none())
      }
    }

    Connected(Error(reason)) -> #(
      Model(..model, status: Failed(reason), last_error: Some(reason)),
      effect.none(),
    )

Field note Hand a rendering surface to FFI, and bootstrap on Connected ↓

Define the board’s fields

The schema lists the shared fields in a document. This board has a title string and two maps named notes and votes. A named shared map is called a channel.

Keep these names in one file instead of repeating strings throughout the app. The BoardDocument type also prevents code from using one of these fields with a different kind of document. For the full picture of what a field can be — a plain value, a handle to a nested map, or a handle to another channel kind entirely — see Foundations · schemas and fields.

src/retro_tutorial_lustre/document_schema.gleam

//// Typed schema for the tutorial retro board.
////
//// The root map carries one field (`title`) and two nested OR-map channels.
//// `notes` uses RegisterMode for whole-note writes. `votes` uses TallyMode
//// for per-note signed tallies.

import gleam/dynamic/decode
import gleam/json
import watershed/schema.{type ChannelField, type Field, type OrMapChannel}

/// Phantom tag for the root map.
pub type BoardDocument

/// The board title shown in the header.
pub fn title() -> Field(BoardDocument, String) {
  schema.field("title", json.string, decode.string)
}

/// Note id → JSON note string in RegisterMode.
pub fn notes() -> ChannelField(BoardDocument, OrMapChannel) {
  schema.channel_field("notes")
}

/// Note id → signed tally in TallyMode.
pub fn votes() -> ChannelField(BoardDocument, OrMapChannel) {
  schema.channel_field("votes")
}

Create the shared maps

A new document starts empty. ensure_or_map creates a map if the field is empty, or adopts whichever one is already there. Open a new board from two tabs at once and both calls still resolve to an attached map — the field settles on one handle without a leader — but the two calls aren't promised to hand back the same map in the same tick. See Foundations · starting a document for the concurrent-bootstrap details.

src/retro_tutorial_lustre.gleam

fn bootstrap_effect(
  document: Document(document_schema.BoardDocument),
) -> Effect(Msg) {
  let root = watershed.root_typed(document)
  effect.batch([
    watershed_lustre.ensure_field(root, document_schema.title(), "Sprint retro"),
    watershed_lustre.ensure_or_map(
      document,
      root,
      document_schema.notes(),
      or_map_kernel.RegisterMode,
      EnsuredNotes,
    ),
    watershed_lustre.ensure_or_map(
      document,
      root,
      document_schema.votes(),
      or_map_kernel.TallyMode,
      EnsuredVotes,
    ),
    watershed_lustre.subscribe(watershed.root(document), fn(_event) {
      SharedChanged
    }),
  ])
}

RegisterMode stores one string per key for notes. TallyMode stores one signed number per key for votes. You choose the mode when you create the channel.

Field note Seed idempotently with Claims ↓

Build the board only when both channels are ready

Each call to ensure_or_map hands its map back separately, and the two don’t arrive together. Build the board as soon as one shows up and you’d get shared state with one channel missing, so the app holds both results in a PendingShared pair and doesn’t assemble the board until both are in. Until then, the shell still renders and its status line reports channels 1/2.

fn assemble(model: Model) -> #(Model, Effect(Msg)) {
  case model.shared, model.pending {
    None, PendingShared(Some(notes), Some(votes)) -> {
      let shared = SharedState(notes:, votes:)
      let model = snapshot(Model(..model, shared: Some(shared)))
      #(
        model,
        effect.batch([
          watershed_lustre.subscribe_or_map(shared.notes, fn(_) {
            SharedChanged
          }),
          watershed_lustre.subscribe_or_map(shared.votes, fn(_) {
            SharedChanged
          }),
        ]),
      )
    }
    _, _ -> #(model, effect.none())
  }
}

The subscriptions come last, and they matter more than they look. A subscription only tells you about changes that happen after you subscribe — it never replays what’s already in the map. That’s why assemble takes a snapshot of the current state in the same breath that it subscribes. Skip the snapshot, and the board boots up empty and stays empty until someone else happens to type.

One message for both origins. A local edit and a remote edit both end in the same snapshot re-read. The local branch calls it after writing; a remote change reaches it through SharedChanged. Keeping one snapshot path prevents the two origins from producing different board state.

Deepening: the same thing on the BEAM

watershed is built BEAM-first — the BEAM is the Erlang virtual machine that Gleam also targets on the server — and the same core logic compiles to the browser too. The map operations are identical either way; only the connecting and subscribing look different, because the two runtimes work differently.

import gleam/erlang/process.{type Subject}
import watershed/map_kernel
import watershed_beam

// Blocks until the document's full history has replayed locally, then
// hands back the document — fine inside an OTP process, but it would
// freeze everything in a browser tab.
let assert Ok(doc) =
  watershed_beam.connect(
    host: "127.0.0.1",
    port: 4000,
    tenant: "dev-tenant",
    document: "retro-tutorial",
    token: token,
    user_id: "ada",
  )

// Root-map changes arrive as a Subject you select on, not as a callback.
let events: Subject(map_kernel.MapEvent) =
  watershed_beam.subscribe(watershed_beam.root(doc))

watershed_beam · for comparison only, the tutorial stays in the browser

On the BEAM, there’s no two-message dance to design around: connect simply pauses your code until the document has fully caught up, so by the time you’re holding a handle, it’s already synced. That’s the difference that changes how you design the connection, and it’s why the browser path above needs the two-message pattern: JavaScript can’t pause and wait the way an OTP process (an Erlang-style lightweight process) can.

Field note One shared core, two runtimes ↓

Run it

Here is the project’s source and test layout, so you know where each later sheet puts its file. The board and note modules arrive in step 02; the app shell already imports them, so use the complete example and read the guide forward.

retro_tutorial_lustre/
├── gleam.toml
├── package.json
├── pnpm-workspace.yaml
├── build.mjs
├── index.html
├── src/
│   ├── retro_tutorial_lustre.gleam ← every sheet
│   └── retro_tutorial_lustre/
│       ├── document_schema.gleam   ← this sheet
│       ├── note.gleam              ← step 02
│       └── board.gleam             ← step 02
└── test/
    ├── retro_tutorial_lustre_test.gleam
    ├── note_codec_test.gleam
    ├── board_test.gleam
    └── convergence_test.gleam      ← step 06

watershed is not published to Hex yet, so the two path dependencies resolve because this example lives in examples/ inside a watershed checkout. In your own project take both as git dependencies pinned to a commit; the README gives the exact lines.

gleam.toml

name = "retro_tutorial_lustre"
version = "1.0.0"
description = "Small retro board tutorial example for watershed Lustre"
licences = ["MIT"]
gleam = ">= 1.7.0"
target = "javascript"

[dependencies]
gleam_stdlib = ">= 0.62.0 and < 2.0.0"
gleam_json = ">= 3.0.0 and < 4.0.0"
gleam_javascript = ">= 1.0.0 and < 2.0.0"
lustre = ">= 5.0.0 and < 6.0.0"
watershed = { path = "../.." }
watershed_lustre = { path = "../../watershed_lustre" }

[dev-dependencies]
gleeunit = ">= 1.0.0 and < 3.0.0"

Start a floodgate development server on port 4000 with just integration-up, then:

cd examples/retro_tutorial_lustre
pnpm install
pnpm build
pnpm serve      # http://localhost:8080

You can now

Open http://localhost:8080 and read the status line: connected · board ready · 0 notes. The document is live and both channels are attached — there is just nothing in them yet. Step 02 fixes that.

How the examples do it

Each note connects an implementation practice to a checked-in example. Open it for the code and reasoning.

Treat the server as an optional decorator

A peer-to-peer app may use a relay for durability, but it should not need one to start.

The clap counter has no sequencer, tenant, or token. If the URL names a relay, one function adds it to the peer-to-peer config.

The app still becomes ready when that relay is down. It reports the outage instead of turning an optional service into a hidden requirement.

src/clap_counter_lustre.gleam

let config =
  crdt_js.config(
    room_id: room,
    replica_label: "tab",
    compatibility_tag: compatibility,
    root: p2p.pn_counter_root(),
    signaling: signaling,
  )
  |> crdt_js.with_ice_servers(ice_servers())
  |> with_relay

/// Attach the optional relay named by `?relay=`, and nothing at all
/// without one. The policy stays `Auto` either way: readiness never waits
/// for a relay, so a URL pointing at a service that is down costs a
/// status line and no claps.
fn with_relay(
  config: crdt_js.Config(PnCounterChannel),
) -> crdt_js.Config(PnCounterChannel) {
  case query(relay_param, "") {
    "" -> config
    url -> crdt_js.with_sequencer(config, crdt_js.sequencer(url))
  }
}

Clap counter

Demonstrated by Clap counter Source ↗

One shared core, two runtimes

Keep the shared core portable by connecting to the same document from the BEAM and the browser.

dice_cli and dice_lustre edit the same floodgate document. They share the kernel, wire format, and runtime core; only the outer program changes. The browser uses an MVU update function, while the BEAM client uses a receive loop.

The CLI connects to 127.0.0.1 instead of localhost. Erlang's resolver stalls about eight seconds on the AAAA lookup, long enough for the server to drop the connection.

src/dice_cli.gleam

fn event_loop(
  map: watershed_beam.SharedMap,
  selector: process.Selector(CliMsg),
  roll_due: process.Subject(Nil),
) -> Nil {
  case process.selector_receive_forever(selector) {
    MapChanged(event) -> {
      print_event(event)
      print_snapshot(map)
      event_loop(map, selector, roll_due)
    }
    RollDue -> {
      roll(map)
      schedule_roll(roll_due, roll_interval_milliseconds)
      event_loop(map, selector, roll_due)
    }
  }
}

Dice CLI

Demonstrated by Dice CLI Source ↗

Sample diagnostics on every event

Put the runtime's diagnostics on screen before you debug synchronization.

The smallest browser example updates one diagnostics line after every event. It shows the connection phase, client id, sequence numbers, queued operations, and resubmit checkpoint.

Those values tell you where to look. A stuck in_flight count, a growing buffer, and a connection that never reaches synced point to different problems that application state cannot show.

src/dice_lustre.gleam

fn diagnostic_line(diagnostics: watershed.Diagnostics) -> String {
  "phase="
  <> diagnostics.phase
  <> " client="
  <> option.unwrap(diagnostics.client_id, "none")
  <> " sn="
  <> option_int(diagnostics.last_seen_sequence_number)
  <> " next_csn="
  <> option_int(diagnostics.next_client_sequence_number)
  <> " in_flight="
  <> int.to_string(diagnostics.in_flight_count)
  <> " buffered="
  <> int.to_string(diagnostics.buffered_out_of_order_count)
  <> " resubmit_at="
  <> option_int(diagnostics.resubmit_checkpoint)
  <> " synced="
  <> bool_to_string(diagnostics.synced)
}

Collaborative dice

Demonstrated by Collaborative dice Source ↗

Hand a rendering surface to FFI, and bootstrap on Connected

Let an FFI module own the canvas pixels, and create shared channels only after the connection opens.

Lustre renders an empty canvas; an FFI module owns its byte buffer and 2D context. The view keeps the canvas size fixed because changing it erases the pixels. The FFI module looks up the context when it needs it, so mount order does not need another effect.

The app creates its shared channel after Connected, not after GotHandle. A handle can resolve before the connection is ready, which would leave the app painting a canvas that no peer can see.

src/pixel_canvas_lustre.gleam

Connected(Ok(_)) ->
  case model.document {
    None -> #(Model(..model, status: Ready), effect.none())
    Some(document) -> {
      let #(canvas, canvas_effect) =
        component.init(document, watershed.root_typed(document))
      #(
        Model(..model, status: Ready, canvas: Some(canvas)),
        effect.map(canvas_effect, Canvas),
      )
    }
  }
Connected(Error(reason)) -> #(
  Model(..model, status: Failed(reason), last_error: Some(reason)),
  effect.none(),
)

Pixel canvas

Demonstrated by Pixel canvas Source ↗

Panels take a TypedMap, never a root

Give a reusable component a typed map, whether that map is a root or a child. Keep document-wide effects in the shell.

The showcase mounts four examples as panels in one document. Each panel accepts a TypedMap and cannot tell whether it received the root or a child map. The shell creates one child field for each panel.

The shell also owns presence, offline mode, and summary policy because they affect the whole document. If panels started their own copies, presence messages could mix and the panels could compete over one shared setting.

src/showcase_lustre.gleam

/// Every tab runs this unconditionally. `ensure_child` creates a map only if
/// the key is absent, so two tabs opening a *cold* document can both create one
/// and LWW settles a single handle — the loser is orphaned before anybody has
/// interacted with it, and every tab converges on the same four handles.
fn bootstrap_effect(document: Document(document_schema.Showcase)) -> Effect(Msg) {
  let root = watershed.root_typed(document)
  effect.batch([
    watershed_lustre.auto_summarize(
      document: document,
      policy: summary_policy.policy()
        |> summary_policy.with_threshold(summary_threshold),
    ),
    watershed_lustre.ensure_child(document, root, document_schema.text(), EnsuredText),
    watershed_lustre.ensure_child(
      document,
      root,
      document_schema.playlist(),
      EnsuredPlaylist,
    ),
    watershed_lustre.ensure_child(document, root, document_schema.sudoku(), EnsuredSudoku),
    watershed_lustre.ensure_child(document, root, document_schema.canvas(), EnsuredCanvas),
  ])
}

Nested app showcase

Demonstrated by Nested app showcase Source ↗

Seed idempotently with Claims

Let every client seed the same initial values through first-writer-wins claims.

Every Sudoku client runs the same loop over the given cells. claim_once keeps the first value for each cell and ignores later attempts, so the clients settle on one puzzle without electing an initializer.

Use this for initial values that must be written once. The shared structure settles duplicate work, so the app needs no separate setup protocol.

src/sudoku_lustre/component.gleam

fn seed_givens(
  claims: Claims,
  active_puzzle: Puzzle,
  row: Int,
  column: Int,
) -> Nil {
  case row >= 9 {
    True -> Nil
    False -> {
      let given = puzzle.given_at(active_puzzle, row, column)
      case given > 0 {
        True -> {
          let _ =
            watershed.claim_once(claims, cell_key(row, column), json.int(given))
          Nil
        }
        False -> Nil
      }
      case column == 8 {
        True -> seed_givens(claims, active_puzzle, row + 1, 0)
        False -> seed_givens(claims, active_puzzle, row, column + 1)
      }
    }
  }
}

Collaborative Sudoku

Demonstrated by Collaborative Sudoku Source ↗