Repository navigation
refactor(certification): write the canonical signing payload instead of serializing it - #589
Merged
Merged
Conversation
…of serializing it
The canonical payload is the message every certification signature -- HMAC and
Ed25519, v1 and v2 -- is computed over, so the bytes must be produced the same way
in every publish mode a consumer can choose. They were produced by reflection-based
`JsonSerializer.Serialize`: the v1 lane over an anonymous type, the v2 lane over a
private positional record. Reflection-based serialization does not survive trimming
or ahead-of-time publishing. Measured in the devtest container across the publish
matrix on a self-contained linux-x64 publish, the same source produced correct
bytes, a throw, or an empty object depending only on the target framework and the
trim settings: under `PublishAot` with defaults and under `PublishTrimmed` with
either `TrimMode` both lanes throw `InvalidOperationException` (reflection-based
serialization disabled); on net10.0 with `TrimMode=full` and
`JsonSerializerIsReflectionEnabledByDefault=true` the v1 anonymous type serialized
to `{}` while the named v2 record came out correct; on net8.0 with `TrimMode=full`
and reflection re-enabled both lanes throw `NotSupportedException` about
constructor parameter names ILLink had trimmed. Under a trimmed publish the payload
could serialize to an empty object, and bytes that back a signature must never be
silently empty.
`BuildPayload` now writes the payload field by field with `Utf8JsonWriter`. The
byte order is the statement order in one method and the names are constants read
by both the writer and the shape guard, so the canonical form is a property of this
file rather than of whichever `System.Text.Json` build a consumer resolves -- the
package ships netstandard2.0, net8.0 and net10.0, which are three different
serializer builds behind one "canonical bytes" claim. `Utf8JsonWriter` still does
the encoding, deliberately: its defaults are compact output and the default
JavaScript encoder, and `WriteNumber(double)` keeps whatever decimal form the
target produces, so each target's existing behaviour is reproduced rather than
replaced. Choosing one decimal form would be a different change and would move
signatures already written.
BYTE-NEUTRAL. `canonical-payloads.golden.json` is unchanged and is the acceptance
test: all five cases -- both lanes populated, both lanes minimal, and the escaping
case -- are byte-identical, byte-length-identical and SHA-256-identical to the
corpus. #584 pinned those bytes and #585 fixed the lane selection, which is what
makes this change verifiable by construction rather than by inspection.
`EnsureCanonical` is unchanged and still called on the writer's output. It is no
longer detecting a serializer degrading under a publish mode -- no input to the
writer can produce another name sequence -- so it is now a post-condition on the
writer, which is exactly what a mistyped name or a swapped pair of write statements
would trip. Two names are worth the guard on their own: camel casing lowercases a
leading run, so the correct spellings are `brickId` and `ed25519PublicKey`, and a
one-character slip in either changes every signature in that lane.
Two behaviour notes. Nulls, sort orders, structural exclusions, the `"O"`-format
UTC timestamp string and the verbatim ordinal-sorted proposer parameter keys are
all preserved exactly. A proposer that supplies the same parameter key twice now
raises `CanonicalPayloadException` rather than the `ArgumentException` the removed
intermediate dictionary raised: `Utf8JsonWriter` does not reject a duplicate
property name, and a payload carrying one describes no single record. The new
vehicle is the one `VerifySignature` already catches, so verification refuses
instead of throwing into its host -- which is the documented contract everywhere
else on that path. Reaching it needs a custom `IReadOnlyDictionary`; a real
`Dictionary` cannot produce it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`CompositionCertificationRecordSigner` produces the message every composition admission signature is computed over -- `CompositionCertificationGate` mints one for every admitted composition, and `CertifiedCompositionRegistry` verifies it -- and nothing pinned those bytes. The brick lane has had a byte-for-byte corpus since #584; this lane could change shape, order or spelling and the only thing that would notice is a stored certificate failing to verify some time after the change shipped. `composition-payloads.golden.json` sits beside the brick corpus in the same shape and with the same regeneration ritual: a populated record, a minimal one, and an escaping case, each pinning the payload string, its UTF-8 byte length and its SHA-256 separately, so a change a string comparison could normalise away still fails. Doubles are restricted to exactly representable values (0.5 and 0.25) for the reason the brick corpus states. The populated fixture supplies `signature` and `gate` as sentinels, so the structural exclusion is asserted rather than assumed. `BuildPayload` becomes internal so the pin can observe the bytes themselves. Going through a signature instead would report "the signature moved" rather than which byte did, and `Ashlar.Infrastructure` already grants `InternalsVisibleTo` to this test assembly. Pinned against the current implementation, unchanged: all three cases pass as-is. That is the point of landing it first -- the corpus records what this signer emits today, so a later change to how it emits has an oracle rather than an opinion. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ializing it
Same change as the brick lane, on the signer `CompositionCertificationGate` uses for
every admitted composition. `BuildPayload` serialized a twelve-member anonymous type
through a freshly allocated `JsonSerializerOptions` on every call. Reflection-based
serialization does not survive trimming or ahead-of-time publishing, and an
anonymous type is the shape that fares worst: measured on net10.0 with
`TrimMode=full` and reflection left enabled, an anonymous type serialized to the
empty object `{}` where a named record came out correct. Under a trimmed publish
the payload could serialize to an empty object, and bytes that back a signature must
never be silently empty.
The payload is now written field by field with `Utf8JsonWriter`, from constants read
by both the writer and the shape check. `Utf8JsonWriter` still does the encoding, so
compact output, the default JavaScript encoder, the `"O"`-format UTC timestamp
string, the ordinal-sorted mutant id lists, the literal nulls and each target's own
decimal form for `CompositionEscapeRate` are all reproduced rather than replaced.
BYTE-NEUTRAL against the corpus landed in the previous commit: all three cases --
populated, minimal and escaping -- are byte-identical, byte-length-identical and
SHA-256-identical, and `Sign` still computes the HMAC over exactly those bytes.
The payload now carries a post-condition of its own, matching the brick lane: the
emitted bytes are re-parsed with `JsonDocument` -- reflection-free, so it holds
wherever the writer holds -- and must carry the exact twelve-name sequence in order.
No record can make the writer produce another sequence, so what it catches is a
mistyped name or a swapped pair of write statements, which would re-shape the signed
payload silently. `Verify` gains a `catch (CanonicalPayloadException)` beside its
existing `catch (FormatException)`, so this lane refuses the way the brick lane
already does instead of throwing into the host; signing still propagates, because a
loud failure at mint time is the right outcome for bytes nobody can vouch for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`StateLogVerifier` recomputes `CertifiedTransitionBuilder.ComputeEntryHash` for every entry in an attested state log and refuses the log when it does not match the stored hash, so those bytes decide whether a log written yesterday still verifies today -- the same role the canonical signing payload plays for a certificate. Nothing pinned them. The only other in-repo computation of them is a re-typed copy inside `AttestedStateLogBindingTests`, which would drift with the production code rather than catch it drifting, and `scripts/ns20-canonical-bytes-probe.sh` exercises the builder end to end but pins a verdict, not bytes: it would still pass if the builder and the verifier changed together. `transition-entry-hashes.golden.json` pins three cases -- the genesis shape (an empty `prevEntryHash`, which must be written as an empty string rather than omitted), a fully linked entry, and an escaping case -- each with the payload string, its UTF-8 byte length, its SHA-256 and the resulting Base64 `entryHash`. The fixtures use real Base64 hashes on purpose: the default JavaScript encoder escapes `+` as a six-character sequence while leaving `/` and `=` alone, which is a property of the real inputs this hash is computed over and was previously pinned nowhere. `BuildCanonicalPayload` is extracted from `ComputeEntryHash` and made internal so the pin can observe the bytes; the public API exposes only their hash, and a pin that could only see the hash would report "the hash moved" rather than which byte moved it. `Ashlar.Certification.State` gains `InternalsVisibleTo` for the test assembly, matching `Ashlar.Certification.Contracts`. Pinned against the current implementation, unchanged: the extraction is a move, and all three cases pass as-is on both halves -- payload bytes and public entry hash. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ead of serializing it Same change as the two certification signers, on the last of the three canonical-bytes producers named in this work. `BuildCanonicalPayload` serialized a five-member anonymous type through a freshly allocated `JsonSerializerOptions` on every call, and `StateLogVerifier` recomputes it for every entry in every attested state log it checks. Reflection-based serialization does not survive trimming or ahead-of-time publishing, and an anonymous type is the shape that fares worst under it: under a trimmed publish the payload could serialize to an empty object, and every entry hash in every log would then be the hash of the same empty object. The payload is now written field by field with `Utf8JsonWriter`. `Ashlar.Certification.State` ships netstandard2.0, net8.0 and net10.0, so the same source runs against three different `System.Text.Json` builds; written this way the byte order is the statement order in one method. `Utf8JsonWriter` still does the encoding, so the default JavaScript encoder still escapes the `+` in a Base64 hash and still leaves `/` and `=` alone. BYTE-NEUTRAL against the corpus landed in the previous commit: all three cases are byte-identical, byte-length-identical and SHA-256-identical, and the public `ComputeEntryHash` returns the same Base64 hash for each. `AttestedStateLogBindingTests` passes unchanged. No shape post-condition here, deliberately, unlike the two signers. Their callers have a refusal contract that `CanonicalPayloadException` fits -- signing propagates, verification returns false -- while `StateLogVerifier` answers with a refusal code and must not throw into its host, and it compares the hash rather than the payload, so there is no place for the exception to land. The byte pin is the check for this site. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…pus cannot state The golden corpora pin the bytes for eleven records across three canonical-bytes producers, which is what makes their shape, order and escaping a fact rather than an intention. A corpus entry cannot hold the inputs where a writer and a serializer could quietly disagree, so those are asserted directly. `CanonicalPayloadEmitterTests` covers: the timestamp is the round-trip UTC string and not `Utf8JsonWriter`'s own `DateTimeOffset` form (that overload looks like the obvious call and emits `+00:00` with trailing fractional zeros trimmed, so the fixture uses a `+02:00` offset to pin the conversion beside the format); a null inside a mutant id list is written as the literal null in ordinal sort position rather than dropped; a proposer with no parameters is an empty object where no proposer at all is the literal null, because those are different records and must not share a signature; parameter keys are emitted verbatim in ordinal order, uppercase before lowercase; a repeated parameter key is refused rather than written twice, and refused as a verification failure (`payload-not-canonical`) rather than an exception in the host; and both lanes emit all thirteen and all twenty names for the emptiest record either accepts. `scripts/ns20-canonical-bytes-probe.sh` gains a fourth check: the certified transition entry hash of every case in `transition-entry-hashes.golden.json`, computed on the shipped netstandard2.0 asset under Mono. `StateLogVerifier` recomputes that hash for every entry in an attested state log, so a target that computed it differently would refuse logs the other targets accept. The payload behind it is internal; its Base64 SHA-256 is what an external consumer can observe, and a hash of the bytes is a byte check. The probe stays the reader of checked-in corpora rather than a typist of its own constants. `WitnessFixture.ComputeEntryHash` in `AttestedStateLogBindingTests` keeps serializing an anonymous type, and now says why: the production builder writes the payload field by field, so the two reach the same bytes by different routes and the fixture is a witness rather than a copy. It was a re-typing of the production code before this change and would have drifted with it. The class docs on the golden and guard suites no longer describe the payload as reflection-serialized. The guard is now a post-condition on the writer -- no input can make it emit another name sequence -- which is the reason to keep it, not a reason to delete it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…T, and run them The canonical bytes are produced by code a host may publish however it likes, and `dotnet build` exercises exactly one of those ways. This probe exercises four more: `PublishTrimmed` with `TrimMode=partial`, with `TrimMode=full`, with `TrimMode=full` and `JsonSerializerIsReflectionEnabledByDefault=true`, and `PublishAot` -- each published self-contained `linux-x64` from a consumer authored **outside** the repo, and each one **run**, with the emitted payloads compared against the checked-in corpora. The third configuration is the one that used to emit the empty object rather than throw, which is why it is named rather than folded into the other two. The consumer reads the corpora with `JsonDocument` and hand-written property access. A reader that used the reflection-based deserializer would fail for its own reasons in exactly the configurations under test and would say nothing about the writer. Measured on this branch, in the devtest container (SDK 10.0.400, runtime 10.0.11): all four configurations publish with **zero** `IL`/`AOT`/`TRIM` analysis warnings and all four emit the five golden canonical payloads and the three golden transition entry hashes byte-identically. The publish settings are written into the consumer project rather than passed as `-p:` switches. A `-p:` switch is an MSBuild global property and flows into the referenced projects, whose netstandard2.0 leg then refuses to build at all (NETSDK1124 / NETSDK1207) -- the configuration under test would never be reached. Like `scripts/ns20-canonical-bytes-probe.sh`, this is a repeatable local measurement rather than merge-blocking coverage; no workflow calls it. `CompositionCertificationRecordSigner` is not covered here: it lives in `Ashlar.Infrastructure`, whose dependency graph is not published trimmed or AOT anywhere. Its bytes are pinned by its own corpus and it is written the same way, but this lane does not execute it. Also spells a generic type plainly in a line comment that had it HTML-escaped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…byte pins Seven entries under Unreleased/Changed: what the three canonical-bytes producers did before and what was measured in each publish mode; how they write their payload now and what Utf8JsonWriter is still deciding on purpose; the byte-neutrality evidence across net8.0, net10.0, the netstandard2.0 asset under Mono, the .NET 9 runtime and four trimmed/AOT publishes; the two new corpora and the two probes that read them; what EnsureCanonical means now that no input can trip it; the two narrow behaviour changes; and the canonical-bytes producers this work does not reach. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two places described the payload as a serialized DTO, which it no longer is. SPEC-006's canonical-form list now says the certification payload is written field by field in a fixed order, and why that matters: the order is the order of the write statements and the names are constants the shape guard reads too, so the canonical form is a property of that file rather than of the serializer a consumer resolves. `ci/cert-gate-assertions.md` gains a row for the two canonical-bytes producers that had no byte pin before this work. The file exists so a returning owner facing a red required check can see what the toggle would turn off, and cert-gate now carries the composition payload pin, the transition entry-hash pin and the writer's own decisions alongside the v1/v2 corpus. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…equences The golden corpora pin that the mutant-id lists and the input list come out sorted; they do not pin what sorts them. Ids that read like real ones - "m-1", "m-10", "m-2" - collate identically under ordinal and culture rules, so replacing StringComparer.Ordinal with StringComparer.InvariantCulture at any of the three sorting sites left every certification test green. The comparer decides the order of bytes that back a signature, so it has to be a fact rather than an intention, and the composition corpus was new in this branch: extending it costs nothing. Adds a composition corpus case whose structural mutant ids collate differently under the two rules, and the two cases the brick lane needs in CanonicalPayloadEmitterTests - the mutant-id lists, and the input order, whose entries also pin that kind outranks id rather than the other way round. Each of the three sorting sites now fails exactly one of the three new pins when its comparer is changed, measured rather than assumed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…rpus, and the third input-hash producer Two statements this branch introduced do not hold. scripts/portability/net9-probe.sh does not read transition-entry-hashes.golden.json. It restores the packed certification contracts package, which does not carry CertifiedTransitionBuilder, and this branch does not touch the script. The corpus _readme added in the same commit already says so, so the XML doc on CertifiedTransitionEntryHashGoldenTests and the Content comment in the csproj contradicted a file beside them. Both now name scripts/ns20-canonical-bytes-probe.sh alone. The "Still not covered" entry named SessionEnvironmentInputs and BrickOutputSerializer and read as exhaustive. CertificationGate.BuildInputs is a third producer on the same footing: it serializes request.Witness reflectively and hashes the result into the "witness" CertificationInput, on the live minting path, and CertificationInput entries are written inside the signed v2 payload - so a trimmed publish still moves the signed bytes for that reason as well. WitnessRunner.CanonicalizeObservation joins BrickOutputSerializer in the not-signed half of the list. A disclosure that reads as complete while it is not is worse than no list. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
IanFrelinger
added a commit
that referenced
this pull request
Sep 11, 2026
… and gate it in CI (#592) * fix(certification): emit one canonical decimal form for every double on every target The canonical payload is the message every certification signature is computed over, so the same record must produce the same bytes wherever it is signed or checked. The decimal form of a double did not: WriteNumber(double) delegates it to whichever formatter the target ships, and the three targets Ashlar.Certification.Contracts builds for do not agree. Measured through BuildPayload in the devtest container. The netstandard2.0 asset under Mono 6.12 writes 1/3 as 0.33333333333333331, 0.007 as 0.0070000000000000001 and 1e-7 as 9.9999999999999995E-08, where net8.0 and net10.0 - byte-identical to each other on all 18 values probed - write 0.3333333333333333, 0.007 and 1E-07. Negative zero diverges the other way: net8.0/net10.0 emit -0 and the ns2.0 asset emits 0, because Mono's formatter drops the sign under "R", "G17" and the default alike. Since #589 the payload is written rather than serialized, so the form is a local decision. It is now made: 17 significant digits on the invariant culture, through WriteRawValue with input validation left on, in both the brick emitter and the composition emitter. G17 because round-trip fidelity is not negotiable for bytes that back a signature and 17 digits is the width at which a binary64 always survives the round trip - a width, not a heuristic. All 22 values probed formatted to character-identical text and re-parsed to the identical bit pattern on all three targets. "R" is disqualified on measurement: Mono's "R" is not .NET's (1/3, 1e16 and double.Epsilon all differ), so it would leave the divergence it was meant to close. A fixed format is disqualified on fidelity: F17 of double.Epsilon is a string of zeros. A shortest-round-trip ladder was measured and works, but makes the bytes depend on the parser at every precision as well as the formatter. Both zeros collapse onto 0 deliberately. Forcing -0 makes the writers agree and leaves the readers disagreeing: under Mono both double.Parse("-0") and GetDouble() on {"x":-0} return +0.0, so a record file written with -0 is re-emitted as 0 by an ns2.0 consumer and the two legs compute different bytes from the same record. Collapsing gives up the sign bit of a zero alone (-0.0 == 0.0) and is stable through emit, read and re-emit everywhere. NaN and the infinities are refused as CanonicalPayloadException rather than left to throw ArgumentException out of Utf8JsonWriter, which VerifySignature does not catch and so threw into its host; the trust verifier now reports payload-not-canonical rather than the catch-all payload-unbuildable. No record in this tree changes bytes. All three signing sites reach the signer with escapeRate zero or null, no production code populates Attempts, and the three committed signed records carry "escapeRate": 0 and keep their signatures. The corpora keep every pinned payload, byte length and SHA-256 - G17 of 0, 0.25 and 0.5 is 0, 0.25 and 0.5 on every target - and gain four brick cases and two composition cases carrying the values that were not exactly representable. Verified: 618 certification tests green on net8.0 and net10.0; all nine brick payloads byte-identical and SHA-256-identical on the shipped netstandard2.0 asset under Mono 6.12, and under PublishTrimmed partial, PublishTrimmed full, full with reflection re-enabled and PublishAot on net10.0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci(portability): run the trimmed/AOT canonical-bytes probe as a job, and make it refuse #589 added scripts/portability/trim-aot-canonical-bytes-probe.sh and nothing in CI ran it. `dotnet build` exercises one of the ways a host may publish, and the canonical signing payload and the certified transition entry hash have to be the same bytes under the others - which is why both producers are written field by field rather than serialized. That claim was a local measurement. It is now the trim-aot-canonical-bytes job in the existing Runtime Portability Gate: a four-configuration matrix (trim-partial, trim-full, trim-full-reflection-on, aot) with fail-fast off, each publishing the minimal certification consumer self-contained linux-x64 on net10.0 and executing the published binary against the same two corpora the xunit golden suites read. One job per mode so a single-mode regression names itself in the check list. No second workflow: this is a sibling job with no `needs`, matching the conventions of the net9 lane beside it. The probe gained a negative half first, because promoting it as it stood would have promoted a weak gate: byte equality alone passes against a build whose verifier had been trimmed into unconditional agreement. Each published binary must now also refuse a signature altered by one bit, a signature over other bytes, a content hash that no longer binds, a schema version that selects no payload lane, a proposer supplying one parameter key twice, and each of NaN and the two infinities - and the byte comparison is fed a deliberately mutated expectation so the comparison itself is shown to be live. The empty-corpus guard extends to the new block. Every check line is teed to one log so the job summary re-reads results rather than re-deriving them, and an absent log says so rather than rendering an empty table that reads like a pass. Both paths: lists gained src/Ashlar.Certification.State/** and transition-entry-hashes.golden.json. The consumer references the one and reads the other, and neither was filtered in, so a change to the transition builder or its corpus would not have fired the workflow - the lane would have looked healthy by never running. Advisory, deliberately: not added to required_status_checks, and no continue-on-error anywhere. The step comment repeats why the publish properties live inside the generated consumer csproj and must never become `-p:` switches. Verified in the devtest container on all four configurations: 9 canonical payloads byte-identical, 3 transition entry hashes identical, 12 refusals answered, 0 trim/AOT analysis warnings each; the summary step rendered against a real probe log. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * style(portability): name the bit-flipped signature buffer for what it is The trim/AOT probe's negative half flips one bit of a valid signature and requires the published binary to answer signature-invalid. The local held the altered bytes and was named for a conclusion the check does not draw: the probe observes that one changed bit is rejected, nothing about intent. `altered` is what the surrounding check text already says ("signature altered by one bit"), so the name and the line it prints now agree. No behaviour change; the buffer, the flip and the expected failure code are unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(certification): state the one class the chosen decimal form does not fix 17 significant digits on the invariant culture fixes the WIDTH of a double on every target. It does not fix the rounding of a value that falls exactly halfway between two 17-digit decimals: that tie is still broken by the target's own formatter, and the targets break it in opposite directions. Measured through BuildPayload, 0.500003814697265625 (0.5 + 2^-18, exactly representable, inside escapeRate's documented 0.0-1.0 range) is written 0.50000381469726562 on net8.0 and net10.0 and 0.50000381469726563 on the shipped netstandard2.0 asset under Mono 6.12 - same width, different bytes, and these bytes back a signature. The branch shipped that as "one decimal form on every target" in six places. That reads as a closed property and it is not one, so every place that asserts the parity now says what the width fixes and what it leaves to the formatter: CertificationRecordSigning.WriteNumberOrNull, the composition emitter that copies it, SPEC-006, both corpus _readme strings, CONSUMING.md's netstandard2.0 row and the CHANGELOG. The direction is unchanged and is still a large improvement - 1/3, 0.007, 1e-7, 0.1 and negative zero all diverged before this work and do not now - but a reader computing a signature over an arbitrary double has to know which class is still open. The residual is pinned rather than described. CanonicalDoubleRepresentationTests carries the value and BOTH known forms, asserts they are neighbours at the last digit and nowhere else, and asserts both read back as the identical bit pattern, so the divergence is a byte-parity residual and not a loss of value. It is pinned there rather than in canonical-payloads.golden.json deliberately: a corpus case would turn scripts/ns20-canonical-bytes-probe.sh red over a divergence no producer in this tree can reach, and both _readme strings now say so. Closing the class means deriving the digits from the bit pattern in the emitter instead of asking a formatter for a width; that would move the bytes of every awkward double again, so it is stated rather than slipped in. Three comments that #589 left behind and this branch made false are corrected with it: WritePayload and BuildPayload both still claimed the emitter keeps "whatever decimal form the target produces", and CompositionCanonicalPayload- GoldenTests still sent readers to a corpus _readme for a restriction the corpus no longer has. CONSUMING.md's inserted clause is closed so the row is one sentence again, and the migration header now covers negative zero, which is not a non-zero double and also moves. No canonical bytes change: the emitter is untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci(portability): make the trim/AOT lane read what the published binary printed The lane's green rested on the consumer's exit code alone. The script wrote a log for the job summary and never read it back, so a published binary that returned 0 having checked nothing, or one that printed PROBE FAIL and returned 0 anyway, made the job green - the same hole the net9 lane closed in #587, in a lane that exists precisely because a trimmed or AOT publish can degrade in silence. check_log now requires each run's output to carry PROBE PASS, all three section verdicts, and one OK payload line per case in canonical-payloads.golden .json, counted from the corpus so a corpus that grows without the lane following it fails too; an empty or short log is a failure on its own. Each run is captured to its own ${label}.run.log and then appended to the cumulative log the summary renders, because "one OK line per case" stops meaning anything once four configurations have appended to one file. The per-run logs upload with it. A configuration name matching none of the four used to skip every configuration, leave FAILURES at 0, and exit 0 printing "every configuration emitted the golden bytes" over a zero-byte log - so a typo or a rename in matrix.configuration turned four checks green having published and executed nothing. Verified before the fix: `... probe.sh trim-fulll` exited 0 with an empty log; it now exits 1 naming the four. The count of configurations that actually ran is checked as well, because an unknown name and a label renamed without the list are opposite mistakes and the name check only catches the first. The final line reports how many configurations ran rather than claiming all of them did. Exercised against both mutations that proved the hole: a log with PROBE PASS and no payload lines, and a log saying PROBE FAIL over a zero exit code, are both refused, as are a missing section verdict, a short payload-line count and a non-zero exit with a complete log. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
IanFrelinger
added a commit
that referenced
this pull request
Sep 12, 2026
…udged (#603) Two more places where reflection-based serialization was fail-OPEN on the certification path. #584 and #589 fixed the signing payload; these are the input-hash and determinism producers beside it, and they degrade the same way. 1. CertificationGate.BuildInputs caught NotSupportedException around JsonSerializer.Serialize(request.Witness, ...), logged a warning, and returned request.AdditionalInputs -- minting the certificate with NO WITNESS INPUT AT ALL. The witness input is the record of WHAT WAS JUDGED. Neither Default nor Strict requires it -- they require gate-emitted-artifact and certifier-identity -- so a witness-less record verifies cleanly downstream and no consumer can tell it binds nothing about the specification. The trigger is not exotic: that exception is what the serializer raises when reflection-based serialization is disabled, which is every trimmed and ahead-of-time publish. So the fail-open was reachable by publish mode alone, with one warning in a log nobody reads at admission time. It now refuses at mint time, carrying the serializer failure as the inner exception so an operator can tell a publish-mode problem from a malformed witness. Refusing at mint is the precedent #584 set: signing propagates, verification refuses without throwing into its host. 2. BrickOutputSerializer.ToCanonicalJson feeds an EQUALITY comparison. The determinism leg runs a brick twice and compares the two strings, so two DIFFERENT outputs that both degrade to {} compare equal -- the leg reports deterministic and the certificate records a check that never ran. An equality comparison is the worst possible consumer of a silently degrading serializer. BrickOutput holds arbitrary object values set by brick code, so there is no closed type set a source generator could cover and reflection cannot simply be removed. Instead the shape is established before the bytes are returned. The invariant is the top-level property count: the values are open-world so nothing about them can be asserted, but the NUMBER of keys is fixed by the dictionary the serializer was handed. Re-read with JsonDocument, which is reflection-free and therefore still works in exactly the configurations where the serializer does not. The count comes from the composed payload, not from the output's dictionary, because a brick whose own output key is "summary" collapses onto the summary slot -- taking it from the dictionary would refuse a legitimate output. That is pinned. Measured in the devtest container: * CanonicalOutputShapeGuardTests 9/9, including a positive control and a case proving two different outputs do not serialize alike; * with EnsureCanonicalShape neutered to return its input, 4 of the 9 fail and the 4 controls still pass; * with the gate's catch restored to the old return, the end-to-end test fails -- it drives CertifyAsync with a witness carrying a delegate, which is what reproduces NotSupportedException in a process where reflection still works; * solution build clean, cert-gate 649/649. Still open, deliberately: SessionEnvironmentInputs serializes SandboxSpec and SessionAttestation by reflection for its sandbox-spec and attestation input hashes. Those are fixed types, so a source-generated context would make them trim-safe without changing the bytes -- a separate change, and one that needs its own golden pin. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The canonical bytes that every certification signature is computed over are now written, not reflection-serialized. Completes the work started in #584 (which pinned them) and #585 (which made verdicts agree across targets).
Why
BuildPayloadproduced those bytes with reflection-basedJsonSerializer.Serialize— a 13-member anonymous type for the v1 lane and a 20-member private positional record for v2. Reflection-based serialization does not survive trimming or Native AOT. Measured across the full publish matrix on a self-contained linux-x64 publish, each configuration published and run:TrimMode=full+ reflection enabled{}; v2 correctPublishAot,TrimMode=partial,TrimMode=fullInvalidOperationException: Reflection-based serialization has been disabledTrimMode=full+ reflection enabledNotSupportedException: … contains parameters with null names … trimmed by ILLinkTrimMode=partialAfter this change every one of those configurations produces the correct bytes.
Important scoping finding, stated plainly: since #584, none of those outcomes reached a signature.
EnsureCanonicalre-parses the payload withJsonDocument— reflection-free, so it still works in exactly the publish modes that break the serializer producing the payload — and turns a wrong-shaped payload into a refusal. So the observable behaviour today is refusal in trimmed/AOT modes, and this PR turns those refusals into working bytes. It does not close an opening.What
A hand-written
Utf8JsonWriteremitter, chosen over a source-generatedJsonSerializerContextfor one reason: it uses only API present in the first netstandard2.0System.Text.Json, so it is provably identical on all three target frameworks without depending on a code generator. Both options were measured byte-identical on net8.0/net10.0 in all 12 plain-and-published runs; the tie-break was the ns2.0 leg.Applied to all three reflection-canonical-bytes producers:
CertificationRecordSigning.BuildPayload(both lanes)CompositionCertificationRecordSigner.BuildPayload(12-member anonymous type) — newcomposition-payloads.golden.json, since it had no byte pin at allCertifiedTransitionBuilderentry hash (5-member anonymous type) — newtransition-entry-hashes.golden.json, likewise unpinnedThe
EnsureCanonicalguard is untouched and still runs; with a written emitter it becomes a post-condition rather than a live failure mode, andCanonicalPayloadGuardTestskeeps driving it directly.Byte parity
canonical-payloads.golden.jsonis bit-identical to master — not merely same content, the same git blob (1b753cf1…). That file is the specification; if it had needed changing, the change would have been wrong.All five cases pinned by payload string, UTF-8 byte length and SHA-256 independently, with the same hashes out of every leg: net8.0, net10.0, the netstandard2.0 asset under Mono 6.12, and .NET 9. Plus
CanonicalPayloadEmitterTestscovering what a corpus cannot state — a record with no proposer versus one with an empty proposer are different records and must not share a signature.New
scripts/portability/trim-aot-canonical-bytes-probe.shpublishes a consumer authored outside the repo, self-contained, and runs each published binary in place. One mechanic worth keeping: the publish settings must be written into the consumer csproj, because a-p:switch is an MSBuild global property that flows into the referenced multi-targeted projects and changes what is being tested.Found along the way
WriteNumberValue(-0.0)emits-0on net8.0/net10.0 and0on the netstandard2.0 asset under Mono. AnescapeRateordurationSecondsof-0.0already produces different signed bytes per target today. Pre-existing and unchanged here.0,0.5,0.25), which are identical on every leg. Queue item for its own change.SessionEnvironmentInputs,BrickOutputSerializer.ToCanonicalJson, andCertificationGate.BuildInputs. They feed input hashes and witness comparison rather than signatures.Review
Three lenses, all mutation-checked. Reordering properties, changing a number format, emitting
{}, or dropping a property each fail a pinned test. Two rounds of required edits applied — including one that mattered: the new composition corpus claimed to pin an ordinal sort but did not discriminate ordinal from culture ordering, so a test now pins which comparer orders the sorted sequences.🤖 Generated with Claude Code