Repository navigation
fix(release): the post-push integrity check could never pass, and ran after the push - #672
Conversation
… after the push WHAT WAS WRONG reusable-release-nuget.yml hashes every .nupkg before pushing (step "Render NuGet release manifest", line 87), pushes (line 187), then downloads what nuget.org serves and asserts the SHA-256 matches (line 251). Those bytes are never equal. nuget.org repository-signs on publish, appending a `.signature.p7s` entry to the archive. Measured on the real feed: ashlar.sdk 0.1.2 as served carries that entry (12987 bytes) and hashes to eb2111dc..., which is not the hash of the file that was pushed. The check could only fail. It fails at the worst possible moment. `dotnet nuget push` has already run, and nuget.org permits unlisting but not deletion, so the release is public and unrecallable by the time the step reports. WHY NOBODY NOTICED It has never executed. The v0.1.2 release (run 33867752688) failed one step earlier, on flat-container index lag - 12 attempts x 15s against an index that had not caught up - so the byte comparison was never reached. That earlier step has since had its budget raised to 40 x 15s, which means the next release is the first time this one runs, and it will fail. A check that cannot pass, sitting behind a check that failed, is indistinguishable from a check that works. WHAT IT COMPARES NOW Every archive entry except the signature, by name and uncompressed content (scripts/lib/nupkg_content_digest.py). That is the part publication must not alter and the part a consumer restores. Verified against the served package: signing appends `.signature.p7s` and touches nothing else - `[Content_Types].xml` does not even declare the p7s extension. The signature's presence is now asserted positively and separately, so excluding it from the digest loses nothing. The manifest keeps the raw `sha256` as provenance for the artifact we produced, and adds `contentSha256`. A manifest written by the old script has no `contentSha256`, and the verifier refuses it by name rather than silently passing. Both sides import the same module, so they cannot drift into computing different things and agreeing anyway. The verifier also refuses a run that verified zero packages, which is what an empty manifest or an all-skipped version match would otherwise look like. tests/scripts/nupkg-content-digest.test.sh holds the two properties that have to be true at once: appending a repository signature must NOT change the digest, or every release fails; and changing any packaged byte MUST change it, or nothing is caught. It also asserts that the raw file hash DOES move when a signature is appended - the fact that made the original check impossible - so the reason stays visible. Mutation-checked: removing the signature exclusion, ignoring entry content, and dropping the length prefix are each caught. SECOND HOLE, SAME PUBLISH PATH release-nuget.yml is a workflow_dispatch taking an arbitrary `version` and handing it straight to the same reusable publish job, with none of release.yml's guards. A dispatch could publish packages whose version disagreed with the assemblies inside them, or leave consumer-template pinning a different release. It now runs the same validate job first (SDK pin, assert_dispatch_version_allowed, assert_consumer_pin_matches). The guards are duplicated rather than shared because a reusable workflow cannot gate its own caller. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
8b2d66e to
621c85b
Compare
`lychee (README + docs)` is a required check with NO path filter on pull_request, so it runs on every PR. It fails on two links in docs/handoff/HANDOFF.md: [403] https://claude.ai/code/artifact/a1b2328c-... (at 38:4) [403] https://claude.ai/code/artifact/bcfebaf4-... (at 40:4) They are PRIVATE artifacts from the session that originated the handoff notes. claude.ai answers an unauthenticated fetch with 403 by design, so a link checker can never verify them - it is measuring claude.ai's auth behaviour, not anything about this repository. This is not a regression from this PR. The links have been on master since 2026-08-22 (#362). The workflow's `push` trigger had not re-run over that file since 2026-09-10, and its `pull_request` trigger has no path filter, so the first PR to reach the check inherited a defect that had been sitting on master for five weeks - and it blocks every PR until it is excluded. Excluded rather than deleted: the links are the only pointer to that reference material for whoever can open them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Added one commit unrelated to the release machinery, because it was blocking this PR and every other one.
Those are private Claude artifacts from the session that originated the handoff notes. claude.ai answers an unauthenticated fetch with 403 by design, so a link checker can never verify them — it is measuring claude.ai's auth behaviour, not anything about this repository. Not a regression from this PR. The links have been on master since 2026-08-22 (#362). The workflow's Excluded in |
…se, and I made it worse (#682) An expert panel reviewing the release before the v0.2.0 tag found that the post-push chain I rewrote in #672 was not fixed, only half fixed - and that three structural faults around it would turn any post-push failure into a release that is public but unrecorded. The tag was held on this. All four are fixed here, each proved by a test or a structural check. 1. MY VERIFIER DIED ON THE FIRST 404 - the same index lag that failed v0.1.2. #672 fixed the byte comparison and kept the download line as it was: a bare urllib.request.urlretrieve, no timeout, no retry, no HTTPError handling. nuget.org indexes packages one at a time after a push, and the visibility poll before this step waited for only four ids, so the other ~18 could still 404 on the first request. That raised an uncaught HTTPError after every package was already public. My own header comment named the lag. It now retries retryable statuses (404, 408, 429, 5xx) and transport errors with a timeout, records a NAMED failure instead of raising, and continues so every failure is reported at once. A non-retryable status (403) fails fast. The feed URL is now overridable, both so this can be tested and so a staging feed can be verified by the same code rather than a copy. tests/scripts/nuget-verify-retry.test.sh drives the REAL script against a local HTTP server that serves 404 for a while and then the package. Eight assertions: lag is ridden out, a package that never appears is a named failure and not a traceback, it is retried the configured number of times, the loop continues past it, a 403 fails fast. With the original bare urlretrieve restored the test reproduces exactly the predicted failure - "urllib.error.HTTPError: HTTP Error 404: Not Found" - and fails. Getting the test to run at all surfaced the friction it is about: python's subprocess.run(["bash", ...]) on Windows resolves bare `bash` to System32\bash.exe, the WSL launcher, which fails before the verifier starts. The server received nothing and two assertions "passed" having measured nothing. It now hands python the bash running the test. 2. THE VISIBILITY POLL WAITED FOR FOUR OF ~22 PACKAGES. NUGET_POST_PUSH_VERIFY_PACKAGE_IDS is unset, so it fell back to a hard-coded Hosting.Bundle, Hosting, Sdk, CLI. It now defaults to every id in the release manifest, which already exists at that point; an explicit operator list still overrides it. 3. GHCR AND NUGET PUBLISHED IN PARALLEL, WITH NO EDGE BETWEEN TWO IRREVERSIBLE ACTS. Both `images` and `nuget` needed only `validate`. In the v0.1.2 run the nexo-cli semver retag completed while the NuGet job was still running, and then NuGet failed. `images` now needs `nuget` to have SUCCEEDED. The order is deliberate: GHCR tags can be re-pointed or deleted, nuget.org packages can only be unlisted, so the irreversible side goes first and the second promise is made only once the first is kept. 4. A POST-PUSH FAILURE SKIPPED THE ONLY DURABLE COPY OF WHAT SHIPPED. `github-release` was `if: success()` over `needs: [validate, images, nuget]`. When any post-push step failed it was skipped - empirically `skipped` in run 33867752688 - so the nupkgs and nuget-publish-manifest.json survived only as a 90-day Actions artifact, for packages that are permanent. That is why the v0.1.2 release page was assembled by hand. A step now records that irreversible publication BEGAN, immediately before the first push - not after the last, since the push loops run under -e and a failure half way still leaves some packages public. It is plumbed out as a job and workflow_call output, and `github-release` runs under always() gated on it. The release is created as a DRAFT, so running after a failure publishes nothing, and the draft now opens with a warning when the NuGet job did not succeed, and says plainly when images were not published rather than printing empty tags. The image tags moved from inline ${{ }} in the run: script into env:, which is the safer shape. 5. THE REHEARSAL EXISTED, AND THE RUNBOOK NEVER SAID SO. I told the maintainer no rehearsal was possible because NUGET_PUBLISH_MODE=oidc means staging would publish for real. That was wrong. `0.2.0-rc1` passes both guards - checked against master - and runs the entire post-push chain against the real feed, at the cost of one permanent prerelease no 0.2.0 pin resolves to. It is now step 8 of "Before you tag". WHAT THIS DOES NOT PROVE. Items 3 and 4 change the job graph of the release workflow, which cannot be run locally. They are checked structurally - both workflows parse, the needs and if expressions are the intended ones, the marker precedes both push paths - but a workflow graph is only proved by running it. That is what the rc1 rehearsal is for. Co-authored-by: t <t@t> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The post-push integrity check could never pass, and it runs after the push
reusable-release-nuget.ymlhashes every.nupkgbefore pushing (line 87), pushes (line 187), then downloads what nuget.org serves and asserts the SHA-256 matches (line 251).Those bytes are never equal. nuget.org repository-signs on publish, appending a
.signature.p7sentry. Measured on the real feed:That is not the hash of the file that was pushed. And it fails at the worst moment:
dotnet nuget pushhas already run, and nuget.org permits unlisting but not deletion, so the release is public and unrecallable by the time the step reports.Why nobody noticed
It has never executed. The v0.1.2 release (run 33867752688) failed one step earlier, on flat-container index lag — 12 attempts × 15s against an index that had not caught up. The byte comparison was never reached.
That earlier step has since had its budget raised to 40 × 15s. So the next release is the first time this one runs, and it will fail. A check that cannot pass, sitting behind a check that failed, is indistinguishable from a check that works.
What it compares now
Every archive entry except the signature, by name and uncompressed content (
scripts/lib/nupkg_content_digest.py) — the part publication must not alter, and the part a consumer restores. Verified against the served package: signing appends.signature.p7sand touches nothing else;[Content_Types].xmldoes not even declare the p7s extension.The signature's presence is now asserted positively and separately, so excluding it from the digest loses nothing. The manifest keeps raw
sha256as provenance and addscontentSha256; a manifest written by the old script has nocontentSha256, and the verifier refuses it by name rather than silently passing. Both sides import the same module, so they cannot drift into computing different things and agreeing anyway. A run that verified zero packages is also refused.The test holds two properties that must be true at once
It also asserts that the raw file hash does move when a signature is appended — the fact that made the original check impossible — so the reason stays visible.
Mutation-checked, all three caught: removing the signature exclusion, ignoring entry content, dropping the length prefix. The length prefix initially survived its mutation, so
frame_entrywas extracted as a pure function and the injectivity property is now asserted on chosen bytes rather than left as a claim no test could reach.Second hole, same publish path
release-nuget.ymlis aworkflow_dispatchtaking an arbitraryversionand handing it straight to the same reusable publish job — with none ofrelease.yml's guards. A dispatch could publish packages whose version disagreed with the assemblies inside them, or leaveconsumer-templatepinning a different release.It now runs the same validate job first: SDK pin,
assert_dispatch_version_allowed,assert_consumer_pin_matches. The guards are duplicated rather than shared because a reusable workflow cannot gate its own caller, and a guard the caller may skip is not a guard.Not changed
ci/published-versionreads0.1.2and that is correct — it names what is on nuget.org, not what is being cut (consumer-template/CONSUMING.md:5).release.yml's guard deliberately requires it to move on the release commit. The gap is that the only written tag procedure (CHANGELOG.md:7) does not mention that step; that is a changelog fix, kept out of this PR so the two do not conflict.🤖 Generated with Claude Code