Repository navigation
fix(#3461): the publication pointer never moves BACKWARDS - #4273
Conversation
#4269 put `prebuilt-bundles/<identity>/plugins` on the generation layout. Copilot's review of it found the one thing that layout can get silently wrong and `flat` could not, and it is real: under `flat` two overlapping publications write ONE directory, so the byte-level postcondition finds the other run's bytes and REFUSES; under `generation` they write disjoint directories, there is no mix to refuse, and whichever run finishes LAST moves `_current`. When that run carries OLDER content, every reader is handed a publication that is complete, sealed, self-consistent and out of date — with nothing red anywhere. The decision-time never-seal-backwards guard cannot see it: it reads the publication that was live when publish_to_target resolved LIVE, ~90 s before the pointer is written, so a sibling that sealed in between is invisible to it. So `pointer_moved_past_us` asks the question again, immediately before move_pointer: re-resolve `_current` by the readers' own rules, and if it now names a different generation, order that generation's source-commit.txt against this bake's SOURCE_SHA through the same compare API. `ahead` ⇒ do not move the pointer, and do not refresh the flat copy either (writing this run's older bytes there would do to pre-pointer readers exactly what the refusal just declined to do to pointer-following ones). The run still SUCCEEDS — its content is contained in what is live — and is counted as `targets-superseded=N`, printed on every run including zero. A postcondition, not mutual exclusion: there is no lease under either storage group. The window shrinks from a whole publication to one read plus a one-line upload, and losing that race is no worse than `flat` today. An unorderable pair keeps today's behaviour and says so, exactly as the decision-time guard does. Also from that review: - `NodeRepoLaneHostGuard.ThePlatformsPluginsBake_DeclaresTheGenerationLayout` — the harness supplies BAKE_PUBLICATION_LAYOUT directly and can never see the caller, so nothing held the flip in place. - the What's New entry no longer claims the race is gone for every installation: pre-pointer readers still see the in-place overwrite until the flat copy goes. Refs #3461 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
The registry/tag publication can still move back to older content, and the overlap test has an incomplete compatibility-copy assertion.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds a final generation-pointer ordering check to prevent older publications from reclaiming _current.
Changes:
- Adds superseded-publication handling and overlap regression coverage.
- Enforces generation layout for the relevant bake workflow.
- Updates documentation and shell-check allowlisting.
File summaries
| File | Summary |
|---|---|
test/MeshWeaver.Documentation.Test/NodeRepoLaneHostGuard.cs |
Guards the workflow’s generation-layout configuration. |
src/MeshWeaver.Documentation/Data/WhatsNew/2026-09-14-an-update-is-never-read-while-it-is-being-written.md |
Updates user-facing publication behavior documentation. |
src/MeshWeaver.Documentation/Data/Architecture/SealedPublicationGenerations.md |
Documents pointer ordering and its residual races. |
.github/workflow-shell.allow |
Allows the intentional fail-open shell check. |
.github/scripts/test-publish-bake-overlap.py |
Adds overlap regression coverage; its compatibility-copy assertion needs strengthening. |
.github/scripts/publish-bake-bundles.sh |
Adds the final pointer-order check; registry/tag publication still requires superseded-run handling. |
Review details
Suppressed comments (1)
src/MeshWeaver.Documentation/Data/Architecture/SealedPublicationGenerations.md:463
- The architecture heading promises that the pointer only moves forward, but the implementation deliberately fails open when the pointer/marker or compare API is unavailable and still has a check-to-write race after this re-read. The later paragraph acknowledges both cases, so this heading should describe a best-effort ordering guard rather than an unconditional guarantee.
#### 🚨 The pointer only moves FORWARD, and this is the one thing the generation layout could get silently wrong that `flat` could not
Under `flat`, two overlapping publications write ONE directory, so the byte-level postcondition
finds the other run's bytes and **refuses**. Under `generation` they write disjoint directories:
there is no mix to refuse, and whichever run finishes **last** moves `_current`. When that is the
- Files reviewed: 6/6 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| echo superseded >> "$OUTCOMES" | ||
| return 0 |
| check("the FLAT compatibility copy was not refreshed with the older bytes either", | ||
| set(s.bakes_present()) - {"<marker>"} != {"older-content"}, | ||
| f"flat: {denominator(s)} — pre-pointer readers must not be handed what the pointer refused") |
| **And the pointer only ever moves forward.** Two updates can finish out of order — the newer one | ||
| first — and the older one must not then take the pointer back, because that would hand every reader | ||
| a set that is complete, correct-looking and out of date. Before moving the pointer, a publication | ||
| re-checks what is live: if something newer arrived while it was working, it leaves the pointer | ||
| alone, says so by name, and its own set stays on the shelf unused. |
Test Results (shard 0)262 tests 262 ✅ 2m 27s ⏱️ Results for commit 74a6c60. |
Test Results (shard 1)1 449 tests 1 449 ✅ 2m 12s ⏱️ Results for commit 74a6c60. |
Test Results (shard 2)899 tests 899 ✅ 3m 22s ⏱️ Results for commit 74a6c60. |
Test Results (shard 4)1 844 tests 1 652 ✅ 4m 34s ⏱️ Results for commit 74a6c60. |
Test Results (shard 5) 6 files 6 suites 5m 17s ⏱️ Results for commit 74a6c60. |
Test Results (shard 3)1 061 tests 1 059 ✅ 6m 7s ⏱️ Results for commit 74a6c60. |
Test Results 18 files 18 suites 24m 2s ⏱️ Results for commit 74a6c60. |
Refs #3461— the defect Copilot found in #4269, which merged before it could be addressed there (77ec6d4, 05:33:18Z). The prefix is on the generation layout now, so this closes the one hole that layout opened.The defect, and why it is specific to the generation layout
_current.Readers never see a torn directory — they see a complete, sealed, self-consistent publication of older bytes, with nothing red anywhere. That is the failure this whole layout exists to prevent, reached from the other side.
🚨 The existing never-seal-backwards guard cannot see it. It runs in
publish_to_target, against the publication that was live whenLIVEwas resolved — roughly 90 seconds before the pointer is written. A sibling that sealed in between is invisible to it.The fix — ask it again, where it can still matter
pointer_moved_past_us, called immediately beforemove_pointer:_currentby the readers' own rules (resolve_publication_dir, not a second copy of them);source-commit.txtand order it against this bake'sSOURCE_SHAthrough the same compare API the decision-time guard uses;ahead(newer, and containing ours) ⇒ do not move the pointer, and do not refresh the flat copy either — writing this run's older bytes there would do to pre-pointer readers exactly what the pointer refusal just declined to do to pointer-following ones.The run still succeeds: its content is contained in what is live, so there is nothing to report as a failure. Its generation stays on the share, sealed and named by nothing — precisely the state retention is defined over. Counted as
targets-superseded=N, printed on every run including zero, for the same reasontargets-convergedis: a number that only appears when non-zero is not a denominator.🚨 This is a postcondition, not mutual exclusion — the same thing
verify_publicationsays about itself, and for the same reason: there is no lease command under either storage group. What it buys is that the window shrinks from a whole publication (~90 s) to one read plus a one-line upload, and that losing that race is no worse thanflat's behaviour today. An unorderable pair keeps today's behaviour and says so, exactly as the decision-time guard does — refusing on an unanswerable comparison would mean a run whose content cannot be ordered never becomes live at all.Negative control — 4 assertions that fail on
mainand pass hereThe harness had no
ghstub, so both compare-API guards had only ever taken their "could not order" arm.STUB_GHimplements the compare endpoint (ahead/behind/identical/diverged, plus the 404 that made 5 of 25 core re-seals a no-op on 2026-09-13), and the new case drives the ordered pair: the newer content publishes and points while the older run is mid-upload.The case's own two fixture assertions — "both generations exist" and "the newer run really did seal first" — pass on main too, so the case is proven to have reached the path rather than to have failed to set it up.
The other two review findings from #4269
test-publish-bake-overlap.pysuppliesBAKE_PUBLICATION_LAYOUTto the script directly and can never see a caller.NodeRepoLaneHostGuard.ThePlatformsPluginsBake_DeclaresTheGenerationLayoutreads the job block out ofmain-cd.yml. Control: with the caller set back toflat,Failed: 1, Passed: 0; restored,Passed: 7.Verification (each step able to fail)
test-publish-bake-overlap.py(this branch)origin/main's publisherNodeRepoLaneHostGuard+DocumentationLinkIntegrityflat(control)dotnet build test/MeshWeaver.Documentation.Test -c Release -warnaserrorbash -n publish-bake-bundles.shcheck-workflow-shell.pycheck-workflow-timeouts.py/check-workflow-yaml-keys.pyDocs:
Doc/Architecture/SealedPublicationGenerations→ phase 4 gains "The pointer only moves FORWARD" with the mechanism, the postcondition-not-mutual-exclusion statement and the measured control.Exposure while this is open
Core CD publishes ~25
pluginspublications/day, and same-lane concurrent publications were measured 4× in one day (2026-09-10) and 0× in the 09-12→09-13 window. So the window is rare-but-real, and only when the finish order inverts the content order.Pairs-with: none— scripts, a guard test and docs; no public type or member leavessrc/. No interface gains a member. No i18n catalog key changes.🤖 Generated with Claude Code