fix(p2p)!: revamp BLOCK_TXS validations - #23778
Conversation
9032d4e to
92c022c
Compare
92c022c to
2f4a8d9
Compare
| // Verify that the returned tx hashes are a subset of (block tx hashes) U (requested tx hashes). | ||
| const uniqueRequestedHashes = new Set([ | ||
| ...blockTxHashes.map(h => h.toString()), | ||
| ...request.txHashes.map(h => h.toString()), | ||
| ]); |
There was a problem hiding this comment.
Why not just check against the requested ones?
There was a problem hiding this comment.
Ah darn, I did indeed want to check (tx hashes by index) U (explicitly requested tx hashes). will fix in the follow up PR.
| const allowedIndexSet = new Set(intersectIdx); | ||
| const indices = returnedHashes.map(h => hashToIndexInBlock.get(h)); | ||
| const allAllowed = indices.every(idx => idx !== undefined && allowedIndexSet.has(idx)); | ||
| const strictlyIncreasing = indices.every((idx, i) => (i === 0 ? idx !== undefined : idx! > indices[i - 1]!)); |
There was a problem hiding this comment.
Any idea why we were checking for this? Should we keep this validation?
There was a problem hiding this comment.
I think this was wrong, or in any case overspecifying. The returned indices and the returned TXs are not really linked (except that you should not withhold a TX that you have and you have been asked for.
I think we shouldn't care about the order of the returned txs. In short I care
- That you give me transactions I ask for (and not more)
- That you don't withhold
- That if you have the block, the sizes of the indices match
Makes sense? Also I think there was nothing preventing a requester to send a tx hash (explicit) that was not in the block, and then this check would fail IIUC (now it is explicit that the requester can do this).
BEGIN_COMMIT_OVERRIDE chore: deploy next-net and reuse contracts (AztecProtocol#23761) chore: turn on autoscaling (AztecProtocol#23706) chore: rename staging-public to staging (AztecProtocol#23767) chore(p2p): use sync hash for tx validation hashing (AztecProtocol#23768) test(e2e): wait warmup slots in slashing tests (AztecProtocol#23719) feat(api)!: make getTxReceipt the single tx-lookup API (AztecProtocol#23660) fix: cap cloned n_tps fees within sponsored FPC balance (AztecProtocol#23770) fix: protect HA validator Postgres from cluster scale-down (AztecProtocol#23772) refactor: remove non-pipelining sequencer code path (AztecProtocol#23665) feat(archiver): add getL2ToL1MembershipWitness node RPC (AztecProtocol#23646) fix(p2p)!: revamp BLOCK_TXS validations (AztecProtocol#23778) chore: name the bots (AztecProtocol#23795) fix(e2e): ensure BBSync init (AztecProtocol#23793) fix(p2p)!: fix BLOCK_TXS response under proposer equivocation (AztecProtocol#23786) fix: reconnect L1 port-forward after epoch-boundary sleep in n_tps_prove (AztecProtocol#23800) chore: add empty vscode settings for yarn-project (AztecProtocol#23808) fix(sequencer): only warn about missing proposed checkpoint once overdue (AztecProtocol#23807) fix: refresh n_tps fee quotes during sustained benchmark (AztecProtocol#23797) fix(sequencer): enforce build-frame deadlines and align attestation/publish windows (AztecProtocol#23776) END_COMMIT_OVERRIDE
## Motivation `v5-next` was cut from `next` at cbc99df (Jun 1), so PRs merged to `merge-train/spartan` after the cut never flowed into it. This backports all of them (authored by @spalladino and @fcarreiro) to keep v5-next current with the spartan train. ## Approach Each PR is cherry-picked from its squashed merge commit on `merge-train/spartan`, in merge order, preserving the original commit message and PR number — one commit per backported PR. All 11 applied cleanly with no conflicts; patches are identical to the originals (verified via `git patch-id`), and `bootstrap.sh build yarn-project` passes on the result. Labeled `ci-no-squash` to preserve the per-PR commits. ## Backported PRs - #23768 — chore(p2p): use sync hash for tx validation hashing - #23719 — test(e2e): wait warmup slots in slashing tests - #23660 — feat(api)!: make getTxReceipt the single tx-lookup API - #23665 — refactor: remove non-pipelining sequencer code path - #23646 — feat(archiver): add getL2ToL1MembershipWitness node RPC - #23778 — fix(p2p)!: revamp BLOCK_TXS validations - #23786 — fix(p2p)!: fix BLOCK_TXS response under proposer equivocation - #23808 — chore: add empty vscode settings for yarn-project - #23807 — fix(sequencer): only warn about missing proposed checkpoint once overdue - #23776 — fix(sequencer): enforce build-frame deadlines and align attestation/publish windows - #23818 — chore(p2p): BlockTxsRequest comment Note #23660, #23778, and #23786 are breaking changes (node RPC + tx-effect db format, and p2p wire format respectively), as they were on `next`.
Summary
BLOCK_TXSrequest/response validation had a bug that caused us to discard perfectly good transactions.When a peer doesn't have the block (proposal pruned, or never received) but the request carried the full tx hashes, the responder (
reqRespBlockTxsHandler) still matches those hashes against its own tx pool and ships whatever it finds — it just can't produce an availability bitvector for a block it doesn't know about. This is a legitimate "I don't have the block, but here are the txs you asked for by hash" response, not misbehaviour.Previously this case was signalled by setting
archiveRoot = Fr.ZEROon the response, andvalidateRequestedBlockTxsConsistencytreated any response that didn't echo the requested archive root (including the zero case) as a hard failure: it returnedfalse, which routed the response through theINTERNAL_ERRORpath and discarded the returned txs entirely. The intended behaviour is the opposite — we want to use the txs the peer returned and merely mark the peer as "dumb" (it can't serve index-based smart requests), without penalising it.Changes
Drop
archiveRootfromBlockTxsResponse.peerHasBlock()helper that derives the same signal from the availability bitvector: an empty bitvector (length 0) means the peer doesn't have the block. The responder no longer special-cases the archive root.Rework
validateRequestedBlockTxsConsistency. Validation now:block tx hashes ∪ request.txHashes(a tx requested by hash may legitimately not belong to the block being validated);true) when the peer signals it lacks the block (!peerHasBlock()) — the returned txs are still valid and usable, which is the core of the fix;The previous order / strictly-increasing and
maxReturnablechecks are removed; membership plus the advertise-vs-deliver check cover the cases that matter.Move dumb-marking into the smart/dumb decision.
BatchTxRequesterno longer inspects archive roots.decideIfPeerIsSmartmarks a peer dumb (and clears its per-peer data, without penalty) whenever the response signals it lacks the block (!peerHasBlock()); penalisation for genuinely inconsistent responses is left to the validator. The oldhandleArchiveRootMismatchhelper is removed.Tests
peerHasBlock()model.request.txHashesthat is not part of the block is accepted).The regression tests fail against the former behaviour.