Skip to content

refactor(pxe): append only NoteStore and PrivateEventStore - #23785

Merged
mverzilli merged 62 commits into
merge-train/fairies-v5from
martin/pxe-reorg-by-canonicality
Jun 5, 2026
Merged

refactor(pxe): append only NoteStore and PrivateEventStore#23785
mverzilli merged 62 commits into
merge-train/fairies-v5from
martin/pxe-reorg-by-canonicality

Conversation

@mverzilli

@mverzilli mverzilli commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Refactors mainly NoteStore so that discovered nullifiers are modeled as append-only rows instead of mutating stored notes. Note the store isn't 100% append-only: adding scopes to a note still causes a modification. I deemed this less cumbersome than introducing more indexes just to track scopes.

Includes some minor touches to PrivateEventStore mostly to align style, since it was already mostly in the shape we need it to.

Context: this is preliminary work for the "time-travelling pxe db" project. We need reorg semantics to be consistent between stores, dealing with mutation on reorgs makes the whole model harder to reason about.

You'll see a longer than expected commit history: this is because I originally set out to change the way we handle reorgs much more fundamentally, by making the stores tolerate multiple versions of notes and events as long as they are recorded as having been originated from different blocks (which could happen in reorg scenarios). Implementing said approach made me realize we don't win much from the additional complexity: in order to filter out "orphaned" data and eventually being able to reap it, we would need to keep track of height->hash mappings from genesis.

At that point, the delete-on-chain-prune original approach seemed more sensible, which ostensibly reduced the scope of this change to what I describe above.

mverzilli added 30 commits June 1, 2026 21:16
merge-train/fairies had reverted the CanonicalBlockStore foundation this
slice builds on. Resolution: slice-owned PXE storage paths take this branch's
version (re-applying the foundation + reorg work, dropping the revert's stale
pre-foundation artifacts); merge-train's other changes (stdlib, simulator,
txe rpc) are merged in. No merge-train work in slice paths was lost (zero
file overlap).
… stores

Reads are now unconditional: delete-on-prune keeps the note and private-event
stores canonical by construction, so a recorded nullification origin always
reflects the canonical chain and event visibility depends only on the block
range. Removes the CanonicalityCheck interface (and canonicality_check.ts), the
per-store #check field/constructor arg, and flips every store construction to a
single argument. Obsolete pure-canonicality tests are removed; behavioral tests
are retained and adapted.
@mverzilli mverzilli changed the title refactor(pxe): use provenance block canonicality to handle note and event reorgs refactor(pxe): append only NoteStore and PrivateEventStore Jun 3, 2026
@mverzilli
mverzilli requested review from Thunkar and nchamo June 4, 2026 09:23
@mverzilli mverzilli added the ci-full Run all master checks. label Jun 4, 2026
Comment thread yarn-project/pxe/src/storage/note_store/note_store.ts

@Thunkar Thunkar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic work, LGTM. Only real concern is abusing nullifiers as keys, given that we want to eventually support external notes CC @nchamo

@nchamo nchamo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Comment thread yarn-project/pxe/src/storage/note_store/note_store.ts Outdated
Comment thread yarn-project/pxe/src/storage/note_store/note_store.ts
@nchamo

nchamo commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Oh, and don't forget to change the target branch!

@mverzilli
mverzilli changed the base branch from merge-train/fairies to merge-train/fairies-v5 June 5, 2026 09:04
@mverzilli
mverzilli enabled auto-merge (squash) June 5, 2026 09:37
@mverzilli
mverzilli merged commit 30c2f6e into merge-train/fairies-v5 Jun 5, 2026
15 checks passed
@mverzilli
mverzilli deleted the martin/pxe-reorg-by-canonicality branch June 5, 2026 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-full Run all master checks.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants