Skip to content

Restore the explicit refresh option and its compiler-required disclosure (#1759 follow-through) - #1797

Merged
erikdarlingdata merged 2 commits into
devfrom
feature/1759-restore-explicit-refresh-options
Jul 28, 2026
Merged

erikdarlingdata merged 2 commits into
devfrom
feature/1759-restore-explicit-refresh-options

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

Reapplies ad4bea7f, which the coordinator's arbitration accepted and verb-reviewer then technically acked by execution — but which is not on dev, because I armed #1788 before either verdict arrived and merged the reverted shape (b0392f7f).

Nothing on dev is wrong. What landed is the shape verb-reviewer acked at 3c3907f0 and independently verified red by flipping the engine (refresh_newest_first: false, which 2.28.1 accepts — the threat itself, not a simulation). Coverage routing, the disk preflight, newest-first slicing, data-based convergence, the sanity clamp and the behavioural mid-slice pin are all there and green. What is missing is the runtime half.

Mechanically a revert of e7576d9b, so the delta is exactly the accepted one plus the connection hardening that landed on dev after it.

What this restores, and why each part is load-bearing

  • refresh_newest_first is DECLARED, not inherited. The resume story depends on a killed slice leaving its NEWEST batches committed. That is not a GUC — it rides the options jsonb — so omitting the parameter means trusting an undeclared engine default. A declared option can only be broken by an engine ignoring its own documented contract; an undeclared default can flip in a release note. That distinction is the whole of Rollup CAGGs serve only materialized buckets: old windows read empty and raw purges stay held #1759, which shipped on exactly this class of trust.
  • The capability is LATCHED PER RUN, which is what dissolved the compatibility objection the earlier revert was ruled on. My first cut re-attempted the 5-argument form on every slice, so a 2.18–2.20 store would have paid a failed round trip per slice. verb-reviewer measured the fixed shape against a simulated pre-2.21 store: 152 slices, one 42883, one disclosure line, DONE with zero missing buckets. The degraded path does not merely exist — it converges.
  • The disclosure sink is REQUIRED by the compiler. A trailing optional is an invisible decision: the first cut made it optional, both call sites omitted it, and a green suite plus green CI plus a self-review all missed that the degrade was silent. Omitting it is now CS7036 — the enforcement is the watched-red, not a test someone has to remember to write.
  • The test sink fails if it fires, so tests on 2.28.1 cannot quietly take the degraded path while asserting the option is carried.
  • The behavioural mid-slice pin stays. verb-reviewer withdrew its withdrawal: on the declared path the pin is redundant, but on the degraded path no option is passed at all, so that is the one configuration where the premise is still inherited — and it is now demonstrably reachable rather than theoretical.

Testing

Full solution rebuild: 0 warnings, 0 errors. All 36 rollup-backfill tests pass.

One unrelated live test (PayloadDimensionLiveTests.DimensionGc_Defers…) fails on my local rig at connection-open, before any code under test runs. Verified pre-existing rather than assumed: it fails identically on plain dev with this change stashed. verb-reviewer saw the same test in its own shared-store contention family. CI is the authority on it.

Provenance

  • Arbitration accepting the shape: coordinator, this session.
  • Technical ack with measured latch evidence: verb-reviewer at ad4bea7f.
  • Merged-without-it explanation: the ack in hand at arm time was bound to 3c3907f0, and arming a shape nobody had reviewed would have broken the rule the whole night ran on.

🤖 Generated with Claude Code

Reapplies ad4bea7, which the coordinator's arbitration ACCEPTED and
verb-reviewer then technically acked by execution — but which was NOT on dev,
because I armed #1788 before either verdict arrived and merged the reverted
shape (b0392f7) instead. Nothing on dev is wrong; it is the shape the reviewer
acked at 3c3907f and verified red by flipping the engine. What was missing is
the runtime half.

Mechanically a revert of e7576d9, so the delta is exactly the accepted one,
plus the connection hardening that landed on dev after it.

What it restores, and why each part is not optional:

- refresh_newest_first is DECLARED, not inherited. The resume story depends on
  a killed slice leaving its NEWEST batches committed; that is not a GUC, it
  rides the options jsonb, so omitting the parameter means trusting an
  undeclared engine default. A declared option can only be broken by an engine
  ignoring its own documented contract -- an undeclared default can flip in a
  release note.
- The capability is LATCHED PER RUN. This is what dissolved the compatibility
  objection the earlier revert was ruled on: my first cut re-attempted the
  5-argument form on every slice, so a 2.18-2.20 store would have paid a failed
  round trip per slice. verb-reviewer measured the fixed shape on a simulated
  pre-2.21 store: 152 slices, ONE 42883, ONE disclosure line, DONE with zero
  missing buckets. The degraded path does not merely exist, it converges.
- The disclosure sink is REQUIRED by the compiler. A trailing optional is an
  invisible decision: the first cut made it optional, both call sites omitted
  it, and a green suite plus green CI plus a self-review all missed that the
  degrade was silent. Omitting it is now CS7036 -- the enforcement is the
  watched-red, not a test someone must remember.
- The test sink FAILS IF IT FIRES, so tests on 2.28.1 cannot quietly take the
  degraded path while asserting the option is carried.
- The behavioural mid-slice pin STAYS. verb-reviewer withdrew its withdrawal:
  on the declared path the pin is redundant, but on the DEGRADED path no option
  is passed at all, so it is the one configuration where the premise is still
  inherited -- and it is now demonstrably reachable, not theoretical.

Full solution rebuild 0 warnings / 0 errors. Darling suite: all 36 rollup
backfill tests pass. One unrelated live test (PayloadDimensionLiveTests
DimensionGc_Defers...) fails on this local rig at connection-open, BEFORE any
code under test runs -- verified pre-existing by running it on plain dev with
this change stashed, where it fails identically. CI is the authority on it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@erikdarlingdata
erikdarlingdata merged commit 815d0e3 into dev Jul 28, 2026
4 checks passed
@erikdarlingdata
erikdarlingdata deleted the feature/1759-restore-explicit-refresh-options branch July 28, 2026 10:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant