Repository navigation
feat(genesis-writer): emit a portable chain dump and manifest - #595
Open
rickyrombo wants to merge 1 commit into
Open
rickyrombo wants to merge 1 commit into
rickyrombo wants to merge 1 commit into
Conversation
After a successful write, pg_dump the chain database (directory format, parallel) to <data-dir>/chain.dump and write <data-dir>/MANIFEST.json, so the artifact is self-contained instead of a live database operators must seal and export by hand. The dump runs before the writer exits, so nothing else has connected first. --no-dump skips it; the manifest is still written. The manifest records chain_id, end/first-live height, the source snapshot's last indexed old-chain block and new_chain_flush_from_block (= that + 1), genesis sha256 and validator address, pg_dump/server versions, the writer commit, and the per-node identity files not to seed onto a bootstrap node. Pin Postgres versions to the node's bundled major (15) via --target-pg-major: the managed cluster runs exactly that major, and a --dst-dsn server newer than it is refused before writing unless --allow-newer-postgres is given. A pg_dump in range is resolved up front. Run cmd/genesis-writer unit tests in make test-unit, and update ROLLOUT.md (steps 1, 4, 7, 8, 11; Appendices B, C, E) and README for the new flow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rickyrombo
force-pushed
the
feat/genesis-writer-portable-dump
branch
from
October 9, 2026 05:39
1f1a756 to
b36dc0b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The genesis writer now produces a self-contained artifact instead of leaving a live Postgres database for the operator to seal and export.
<data-dir>/chain.dump: after a successful write (after genesis.json, state.db and the index rebuild), the writer runspg_dump --format=directory --jobs=Non the chain DB before it exits, so nothing else has connected first. It dumps tochain.dump.partialand renames on success. It leaves out the writer's owngenesis_writer_progresstable, plus owners and ACLs. On by default;--no-dump(or--dump=false) skips it, and--dump-jobssets the parallelism.<data-dir>/MANIFEST.json:chain_id,end_height,first_live_height,source_chain_id,source_last_indexed_block(MAX(height) in the source'score_indexed_blocksforaudius-mainnet-alpha-beta, read at the start of the run),new_chain_flush_from_block(= source + 1),genesis_sha256,genesis_validator_address,genesis_migration_address,dump.{pg_dump_version, pg_dump_major, server_version, server_major, restore_min_pg_major, size_bytes},writer_commit, andexclude_from_bootstrap(node_key / priv_validator_key / addrbook / priv_validator_state, each with a reason and whether it was present).--target-pg-major, which defaults to 15, the node image's bundled Postgres:--dst-dsn, the writer checks the server's major before writing anything. A server newer than the target is refused unless you pass--allow-newer-postgres, and then it logs loudly.pg_dumpwith server ≤ major ≤ target is resolved up front, so a missing client fails in seconds, not after the run.pg_dump/pg_restorenever get the password on argv; it goes throughPGPASSWORD.--no-dumpruns and the 2026-08-25 artifact.pg_restoreinto a new database before first start, then restore the mediorum tables, and exclude the manifest's identity files.NewChainFlushFromBlock= source last block + 1, because the flusher deletesconfirmed_block < value.make test-unitnow includes./cmd/genesis-writer/.... Before this, no make target ran the package's unit tests. Its DB-backed tests skip withoutETL_TEST_DB_URL, as they already did.The 2026-08-25 artifacts predate this and still need the manual dump. This change is for future writer runs.
Test plan
make test-unitpasses. It includes the newTestManifestDescribesTheArtifact,TestManifestWithoutDumpRecordsNull,TestParsePgToolMajor,TestPostgresVersionChecks(covers the PG17-server-into-PG15-node case) andTestPgDumpKeepsPasswordOffArgv.--allow-newer-postgres.pg_dumpand dumps.pg_restore --list(PG15) reads the archive: format 1.14, indexes and constraints included, progress table excluded.chain.dumpis replaced.cmd/genesis-writermake test, docker stack). It buildsWriterConfigdirectly withoutArtifactDir, so the artifact step is skipped there.🤖 Generated with Claude Code