watershed Collaborative data structures for Gleam

← watershed · Data structures / Maps

Maps

Keyed state, resolved two different ways.

Maps are where most collaborative apps keep their state, and where the choice of conflict model is most visible. watershed’s maps span that choice.

SharedMap resolves each key by server order, following the last-write-wins design used by Fluid Framework. OR-map keeps causal dots per entry so a concurrent write survives a delete (correctness over simplicity when last-write-wins would drop data). SharedDirectory makes SharedMap recursive: folders of keys and nested folders, with a hierarchical identity that survives concurrent creation and delete-then-recreate.

Skip past the interactive demo

Watch maps converge

The structures from this family that fit the shared gauge rig run below on watershed’s Gleam kernels. Structures with dedicated interactions link to their own demos. Pending edits print in magenta; once the sequencer stamps an SN, every replica lands the same ink state. The picker switches among the loaded views, which share one ordered stream.

Nudge a value, race a concurrent write, or stretch the link latency to watch each merge rule hold.

  1. 1 · local edit prints in magenta
  2. 2 · the sequencer stamps it with an SN
  3. 3 · every client lands the same ink state
Data structure shown in all clients

Merge rule: last write wins. Race two writes to the same key and the op the sequencer stamps later overwrites the earlier one, identically on every replica.

Client A

0 pending
Shared map replica on Client A
mill-race 24
kettle-run 61
low-ford 42
sandbags-placed 120
earthwork-balance · yd³ +44
fill Σ
74
cut Σ
30
inspection-count 18
A Σ
0
B Σ
0
C Σ
0
Claims replica on Client A
north-levee
spillway-gate
pump-house Survey
OR-map stockpile ledger replica on Client A
spoil-north 18
borrow-pit-7 -6
wash-fill 12
OR-set field marker roster replica on Client A
north-stake marked
sluice-tag marked
borrow-flag clear
G-set permanent benchmark registry replica on Client A
BM-17 recorded
BM-22 unrecorded
BM-31 unrecorded
2P-set retired marker ledger replica on Client A
stake-3 active
gate-pin unplaced
silt-flag retired
RegisterCollection replica on Client A
register atomic LWW revise
north-bench Survey Survey
gate-setpoint
pump-mode
OrderedCollection replica on Client A
sheet state op
queue grade-stakes, pump-check
held jobs none
PactMap replica on Client A
pact accepted pending op
datum-grid Survey datum
gate-policy
inspection-window
TaskManager replica on Client A
task assigned waiters op
sluice-inspection empty
pump-watch empty
crest-walk empty

Client B

0 pending
Shared map replica on Client B
mill-race 24
kettle-run 61
low-ford 42
sandbags-placed 120
earthwork-balance · yd³ +44
fill Σ
74
cut Σ
30
inspection-count 18
A Σ
0
B Σ
0
C Σ
0
Claims replica on Client B
north-levee
spillway-gate
pump-house Survey
OR-map stockpile ledger replica on Client B
spoil-north 18
borrow-pit-7 -6
wash-fill 12
OR-set field marker roster replica on Client B
north-stake marked
sluice-tag marked
borrow-flag clear
G-set permanent benchmark registry replica on Client B
BM-17 recorded
BM-22 unrecorded
BM-31 unrecorded
2P-set retired marker ledger replica on Client B
stake-3 active
gate-pin unplaced
silt-flag retired
RegisterCollection replica on Client B
register atomic LWW revise
north-bench Survey Survey
gate-setpoint
pump-mode
OrderedCollection replica on Client B
sheet state op
queue grade-stakes, pump-check
held jobs none
PactMap replica on Client B
pact accepted pending op
datum-grid Survey datum
gate-policy
inspection-window
TaskManager replica on Client B
task assigned waiters op
sluice-inspection empty
pump-watch empty
crest-walk empty

Client C

0 pending
Shared map replica on Client C
mill-race 24
kettle-run 61
low-ford 42
sandbags-placed 120
earthwork-balance · yd³ +44
fill Σ
74
cut Σ
30
inspection-count 18
A Σ
0
B Σ
0
C Σ
0
Claims replica on Client C
north-levee
spillway-gate
pump-house Survey
OR-map stockpile ledger replica on Client C
spoil-north 18
borrow-pit-7 -6
wash-fill 12
OR-set field marker roster replica on Client C
north-stake marked
sluice-tag marked
borrow-flag clear
G-set permanent benchmark registry replica on Client C
BM-17 recorded
BM-22 unrecorded
BM-31 unrecorded
2P-set retired marker ledger replica on Client C
stake-3 active
gate-pin unplaced
silt-flag retired
RegisterCollection replica on Client C
register atomic LWW revise
north-bench Survey Survey
gate-setpoint
pump-mode
OrderedCollection replica on Client C
sheet state op
queue grade-stakes, pump-check
held jobs none
PactMap replica on Client C
pact accepted pending op
datum-grid Survey datum
gate-policy
inspection-window
TaskManager replica on Client C
task assigned waiters op
sluice-inspection empty
pump-watch empty
crest-walk empty
Sequencer SN 0

    Loading booting watershed kernels…

    Latency and jitter affect simulated arrival order; animation speed changes playback only. “Ops in flight” counts every hop still travelling: one client → sequencer leg, then one sequencer → replica leg per client.

    SharedMap

    DDS
    map_kernel

    A shared key/value map where the most recent write to a key wins.

    Open the live Sudoku cell demo →

    watershed’s flagship DDS follows Fluid Framework’s SharedMap kernel design. Keys map to JSON values. Each set is sequenced, and for a given key the write with the highest sequence number wins. Its inner set, delete, and clear payloads match the @fluidframework/map operation encoding.

    Concurrent writes resolve deterministically by server order rather than by a merge function. A local write renders immediately; the ack promotes it, and if a higher-SN write to the same key arrives it replaces the value. Reference-generated corpus tests cover map state, events, and convergence. Attach and summary formats remain watershed’s own, so this does not imply drop-in Fluid container interoperability.

    Best for

    • Shared application state and settings objects edited by many clients
    • Learning and testing server-ordered last-write-wins collaboration
    • Key-value collaboration where a clear last-writer-wins rule is acceptable
    Merge rule
    for each key, the most recent write wins, decided by server order
    Optimistic behavior
    your writes show instantly, then lock in once the server confirms them
    Summary shape
    confirmed entries reload with their keys and insertion order intact
    Model
    Distributed data structure: converges by server order

    OrMap

    CRDT
    or_map_kernel

    A map where editing a key and deleting it at once won’t lose the edit.

    An OR-map applies the OR-set’s observed-remove semantics to keyed entries. Each entry records causal dots; removing a key only tombstones the dots it has observed, so a concurrent write to the same key survives a delete (add-wins).

    Values can themselves be additive tallies, which turns the map into a keyed CRDT ledger. In the demo it appears as a stockpile ledger where striking a row hides it and re-opening submits a +0 delta to surface the retained tally.

    Best for

    • Keyed ledgers edited offline or concurrently (stockpiles, inventories, per-key counters)
    • Maps where deleting and concurrently updating a key must not lose the update
    • A CRDT-correct alternative to SharedMap when last-write-wins would drop data
    Merge rule
    delete a key while someone else edits it, and the edit survives (the write wins)
    Optimistic behavior
    deleted rows stay readable until the delete is confirmed
    Summary shape
    entries remember their edit history, not just the current value
    Model
    Conflict-free replicated data type: converges by merge

    SharedDirectory

    DDS
    directory_kernel

    SharedMap with folders: nested groups of keys, each keeping its own identity.

    Open the live SharedDirectory demo →

    SharedDirectory is SharedMap made recursive, modeled after Fluid Framework’s SharedDirectory design. Every folder node has its own last-write-wins key/value store plus a named set of child folders, addressed by absolute path (/surveys, /surveys/intake). Storage resolves exactly like SharedMap: each set is sequenced, highest sequence number wins per key.

    The hard part is hierarchical identity, not storage. A folder can be created by two clients at the same instant, deleted, and recreated under the same path, and every replica must still agree on which folder is which. The kernel models that identity explicitly from creator ids, create-sequence data, and each op’s reference sequence number. A stale op targeting an old instance of a path is ignored; concurrent same-name creates merge into a single folder. A flat map cannot express that.

    Best for

    • Nested, collaboratively-edited state: document trees, project/site hierarchies, scene graphs
    • Studying hierarchical identity and server-ordered folder collaboration
    • Anywhere a flat map’s keys want structure (folders of readings, grouped settings)
    Merge rule
    like SharedMap, but with folders; each folder keeps its identity even if it’s deleted and remade
    Optimistic behavior
    folder and key edits show immediately until the server confirms them
    Summary shape
    the whole folder tree reloads intact
    Model
    Distributed data structure: converges by server order