Skip to content

plan: blob store backups - #809

Closed
dannash100 wants to merge 1 commit into
mainfrom
plan/blob-store-backups
Closed

dannash100 wants to merge 1 commit into
mainfrom
plan/blob-store-backups

Conversation

@dannash100

Copy link
Copy Markdown
Contributor

Tamanu is moving attachment and asset bytes out of the database into a content-addressed blob store on disk. Once it does, a tamanu-postgres backup stops capturing a server's whole state: the database holds the references, the store holds the bytes, and the two have to be backed up and restored consistently.

The behaviour is specified Tamanu-side (specs/blob-storage/backups.md, id BKUP, in the blob store epic). This is the brief for our end, added as a plan rather than folded into BAK because two of its requirements have no mechanism here yet and the design isn't settled. Options are laid out, nothing is chosen.

The two gaps:

  • Ordering within a cycle. The database must be captured before the store, so the store capture is a superset of what the database references. Reversed, a blob admitted between the two is referenced by the restored database and absent from the restored store. But BackupDef has no dependency field and Canopy emits due types with no declared order, so there's no way to express it. Three options in the plan, the honest one being a definition-level ordering field that Canopy respects.
  • Pairing the captures. Restore has to pick the store capture belonging with the database capture. Each run mints its own id, so today that's an operator matching timestamps by eye. The freeze moment we already report is the natural anchor.

Two things that make this easier than it sounds: the store needs no freeze, since blobs are immutable and land by atomic rename, so simple is correct against a live store; and each store capture represents the whole store rather than a delta, so retention can't strand a blob that's still on disk.

Also flags the boundary on restore: reconciling the tree against Tamanu's own registry of what it holds is Tamanu's job, not ours, and neither side should assume the other does it.

@dannash100 dannash100 closed this Aug 6, 2026
@dannash100
dannash100 deleted the plan/blob-store-backups branch August 6, 2026 23:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant