Skip to main content

Offline-first

The product requirement is one sentence: the app behaves identically at the bottom of a crag with no signal as it does on wifi. Not "has an offline mode" — offline is the normal case, and connectivity is the exception that occasionally lets data move.

The rule​

Screens read from on-device SQLite. Nothing else. Network access lives in apps/mobile/src/lib/sync.ts and nowhere else.

A screen that calls api.* directly will hang the moment someone opens it in a car park. The rule is mechanical and easy to review, which is why it is worded that way.

Note the shape: the screen and the syncer both touch SQLite, and they never touch each other.

Two kinds of data​

The protocol treats them differently because their failure modes are different.

Server-ownedUser-owned
WhatCrags, sectors, photos, routes, route linesAscents, wishlist items
Who writesUs, through curation and moderationThe user, on their phone
DirectionPull onlyPush and pull
On conflictServer always wins; there is nothing to loseGuarded — see below
VolumeLarge; scoped to downloaded cragsSmall; all of it, always

Guidebook content is read-mostly and authoritative. A user's logbook is the opposite: small, precious, and created offline. Pretending they are the same thing is how sync engines get complicated.

Pull​

POST /v1/sync/pull
{ "cursors": { "route": "1735689600000:rte_01J…" }, "cragIds": ["crg_01J…"], "limit": 500 }

The client keeps a cursor per kind. The server returns everything after it.

Why the cursor is (updated_at, id) and not just updated_at​

Two records can share a millisecond. With a plain updated_at > ? cursor, the second one is silently skipped forever — the client asks for "after 12:00:00.123", and the row at .123 never comes back.

So the cursor is the pair, and the predicate is the lexicographic comparison:

where updated_at > :cursorUpdatedAt
   or (updated_at = :cursorUpdatedAt and id > :cursorId)
order by updated_at, id

Which is exactly what the (updated_at, id) index on every syncable table is for.

An offset-based cursor would have a different but equally bad failure: insert a row mid-page and everything after it shifts, dropping one record per concurrent write.

A malformed cursor is a 400, not a silent fall back to "send everything". Falling back would turn a client bug into a full table download on every sync.

Tombstones come through​

Soft-deleted rows are returned by pull, not filtered out. That is the only way a device that was offline when a route was removed ever learns about it. The client applies deleted_at and its queries filter on it.

Paging​

The server returns limit + 1 rows, sends limit, and reports hasMore. The client keeps pulling until hasMore is false, up to a cap of 50 pages — a pull that never terminates means the cursor is not advancing, which is a server bug, and hammering it is not the right response.

Push​

POST /v1/sync/push
{ "mutations": [
    { "kind": "ascent", "op": "upsert", "baseUpdatedAt": null, "record": { … } }
] }

Every mutation carries baseUpdatedAt: the updated_at the client last saw for that record.

  • Row does not exist → insert. baseUpdatedAt is null.
  • Row exists and updated_at matches → update.
  • Row exists and updated_at does not match → conflict, with the server's copy attached.
  • record.userId is not the caller → rejected. Never silently, never as a 500.

This is optimistic concurrency, and it is the cheapest thing that surfaces a real conflict instead of quietly overwriting the other phone's edit.

Push before pull​

Deliberate. The outbox is work the user did; stale crag data is not. If a sync is going to be cut short by a dropped connection, the half that ran should be the half that protects the user's data.

The outbox​

create table outbox (
  seq             integer primary key autoincrement,
  kind            text not null,
  op              text not null,
  record_id       text not null,
  base_updated_at integer,
  payload         text not null,
  queued_at       integer not null,
  attempts        integer not null default 0,
  last_error      text
);

seq is autoincrement because ordering within one device is all that matters here — this table never syncs.

Enqueue in the same user action as the local write. Never from a background job: if the local row and the outbox row can diverge, the user sees one thing and the server gets another.

Conflicts, honestly​

Right now: the server's copy wins, and the client discards the local edit silently.

The protocol reports the conflict properly and hands back the server record. The app throws it away. That is a real limitation with a real failure case — edit the same tick on a phone and an iPad while both are offline, and one edit disappears without a word.

It is acceptable today because there is one device per user and no real authentication yet. It is on the roadmap for phase 2, and it is written down here rather than discovered later.

Options when it is time, roughly in order of how much they cost:

  1. Last-write-wins with a notice — cheap, honest, probably enough for a logbook
  2. Field-level merge — ascents are mostly independent fields; a merge is plausible
  3. Interactive resolution — a screen showing both, user picks
  4. CRDTs — rejected for v1 in ADR-0004; the data model does not have the concurrent-editing shape that justifies the complexity

Being offline is not an error​

sync() returns a discriminated union, not a thrown exception:

type SyncOutcome =
  | { status: 'ok'; pulled: number; pushed: number; conflicts: number }
  | { status: 'skipped'; reason: 'offline' }
  | { status: 'error'; message: string };

skipped is a normal Tuesday. It surfaces as a quiet line on the Profile screen and nothing else — no dialog, no red banner, no retry prompt. Every screen already has its data.

Images​

The gap. Photo.storageKey points at an R2 bucket that does not exist yet, so the topo viewer — the actual point of the app — is not built.

Whatever the design ends up being, it has to answer:

  • Download size. A crag's worth of topo photos is the bulk of an offline download. Users need to see the size before they commit to it on a phone plan.
  • Eviction. Phones fill up. Something has to decide what goes when, and the user has to be able to override it.
  • Partial downloads. A crag that is 80% downloaded should be 80% usable, not unusable.

This needs an ADR before any code. See roadmap phase 1.

When sync runs​

Today: on app launch, and when the user taps "Sync now" on the Profile screen.

Not yet: background refresh, on-network-regained, or after any write. All three are wanted; all three have battery implications that deserve more thought than "add a listener".

Testing this​

The API tests in apps/api/test/api.test.ts run inside workerd against a real D1 — not a mock — because the bugs worth catching here are pagination bugs, and a mocked database will happily return whatever you told it to. Covered today:

  • A pull replayed with the returned cursors yields nothing new
  • A malformed cursor is a 400, not a full download
  • A stale baseUpdatedAt produces conflict with the server record attached
  • A mutation for another user's record is rejected

Not covered yet, and the obvious next tests: tombstone propagation, multi-page pulls, and the same-millisecond tie that the composite cursor exists to handle.