refactor(certification): write the canonical signing payload instead of serializing it

CHANGELOG.md

@@ -24,6 +24,13 @@ At release time, move the `[Unreleased]` notes under a new `[X.Y.Z] - YYYY-MM-DD
 - **Every target framework now reaches the same verdict for the same certification record, and a signature a target cannot evaluate is refused rather than skipped.** `Ashlar.Certification.Contracts` ships `netstandard2.0` for external consumers, and that asset has no Ed25519 implementation (NSec ships `lib/net8.0` only). Its `CertificationTrustVerifier` never read `Ed25519Signature`: under options that did not *require* a signature — `Legacy`, or any custom options with `RequireEd25519Signature` false and no pinning — a record carrying an Ed25519 signature that does not verify fell through to the content-hash check and was TRUSTED, while the net8.0 and net10.0 assets refuse the same bytes with `ed25519-signature-invalid`. Measured on the shipped netstandard2.0 asset under Mono against the golden corpus: the two admitted records carrying an Ed25519 placeholder that is not a signature over anything came back TRUSTED under `Legacy` before this change and are refused with the new `ed25519-signature-unverifiable` code after it; the HMAC-only shapes of every admitted record are TRUSTED before and after. The presence test and the key-missing test now sit outside the target-framework fence, so "present" means the same bytes on every target and `ed25519-key-missing` is reported identically; only the evaluation differs, and where it cannot run the record is refused at the same position in the check sequence (after the HMAC, before the content hash), so a multi-fault record reports the same code everywhere. `ed25519-verification-unavailable` keeps its meaning — strictness was requested that this build cannot deliver — and now fires only for an absent signature. **Behaviour change for netstandard2.0 / .NET Framework consumers:** a dual-signed record verified under `Legacy` on that asset was trusted and is now refused; the set of records that asset trusts is a subset of what net8.0 trusts (HMAC-only records), never a contradiction. No in-repo caller is affected: every production caller is net8.0+ and passes `Strict`. `VerifierParityTests` pins the net8.0 side against the same corpus the probe reads, so the subset property is asserted from both sides against one set of bytes.
 - **`StateLogVerifier.Verify` gains an overload that takes `CertificationVerifyOptions`; the existing signature is unchanged and still means `Strict`.** The verifier hardcoded `CertificationVerifyOptions.Strict`, and Strict requires an Ed25519 signature — which the `netstandard2.0` asset of `Ashlar.Certification.Contracts` cannot evaluate. On that asset the state-log verifier therefore could never return a trusted verdict (every transition was `behavior-cert-untrusted` with `ed25519-verification-unavailable` inside), with no configuration escape. The new overload places `options` last and non-optional, because the existing overload takes an optional `string? hmacKey` in the slot a nullable options parameter would otherwise occupy and a literal `null` argument must keep meaning "no key"; the existing signature delegates to it with `Strict`, so no caller changes and no default changes. A null options instance is refused (`verify-options-missing`) in the verifier's own refusal convention rather than defaulted — a verifier that was not told how strict to be must not guess. The overload does not widen what the netstandard2.0 asset can trust: a certificate that carries an Ed25519 signature is refused there under every options instance (see the entry above), so only HMAC-only certificates ever verify on it, and the set it trusts stays a subset of what net8.0 trusts. Measured under Mono on the shipped netstandard2.0 assets with `scripts/ns20-canonical-bytes-probe.sh`: a one-transition log bound to an HMAC-only certificate is refused through the existing signature (`behavior-cert-untrusted: ed25519-verification-unavailable`) and trusted through the overload with `Legacy`; the same log bound to a certificate carrying an Ed25519 signature is refused through the overload with `Legacy` (`ed25519-signature-unverifiable`). `AttestedStateLogBindingTests` pins the net8.0 side: the two signatures yield identical results under Strict, Legacy trusts an HMAC-only log Strict refuses, and null options are refused.
 - **An unknown certification schema version is now an error on both the signing and the verification paths, not a guess.** `CertificationRecordSigning.BuildPayload` selected the v2 payload lane on `SchemaVersion is not null`, so a record stamped 3, 7 or `int.MaxValue` was serialized in the v2 shape with that number emitted verbatim — and, because `MinimumSchemaVersion` only says "at least this new", cleared the floor of 2 that `Default` and `Strict` apply. Only null (v1) and `TrustLoopSchemaVersion` (2) select a lane now, through one `IsKnownSchemaVersion` used by both paths so they cannot disagree. Signing propagates the refusal as a `CanonicalPayloadException` — the same vehicle a degenerate payload uses, loud at mint time — while every verification path refuses without throwing: `VerifySignature` and `CertificationRecordEd25519.VerifySignature` return false, `CertificationRecordSigner.Verify` returns false (so a stored record carrying such a version no longer loads), and `CertificationTrustVerifier` reports a dedicated `schema-version-unknown` code rather than `payload-not-canonical`, because that code is documented as the serializer degrading under trimming or ahead-of-time publishing (a deployment fault) and an unknown version is a record fault. Precedence is floor first, so an explicit 0 or 1 under `Default` still reports `schema-version-below-floor`. **Byte-neutral for every valid record:** the null and 2 branches are untouched, `canonical-payloads.golden.json` is unchanged, and the golden tests pass against it as-is; measured on the netstandard2.0 asset under Mono, all five payloads are still byte-identical and a record stamped 3 or `int.MaxValue` is refused there with the same code. No in-repo writer mints any version other than null or 2 (`CertificationGate`, `LoadRefusalRecord`, and every test fixture were checked); a record with another version, however it came to be, was previously accepted by a floor and is now refused everywhere. `UnknownSchemaVersionTests` pins every path.
+- **The canonical bytes behind every certification signature and every attested state log are now written rather than serialized, so a trimmed or ahead-of-time publish emits the same bytes a plain build does.** The canonical payload is the message every certification signature — HMAC and Ed25519, v1 and v2 — is computed over, and the certified transition entry hash is what `StateLogVerifier` recomputes for every entry in an attested state log. Three producers built them with reflection-based `System.Text.Json`: `CertificationRecordSigning.BuildPayload` (an anonymous type on the v1 lane, a private positional record on v2), `CompositionCertificationRecordSigner.BuildPayload` (a twelve-member anonymous type, live behind `CompositionCertificationGate`), and `CertifiedTransitionBuilder` (a five-member anonymous type). **Reflection-based serialization does not survive trimming or ahead-of-time publishing.** #584 said so and added the shape guard that turns the failure into a refusal, which is the right outcome for bytes nobody can vouch for — but a refusal is still a host that cannot mint or check a certificate. Measured before this change in the devtest container (SDK 10.0.400) on self-contained `linux-x64` publishes that were actually **run**: under `PublishTrimmed` with `TrimMode=partial`, with `TrimMode=full`, and under `PublishAot`, the payload threw `InvalidOperationException` ("Reflection-based serialization has been disabled for this application"); on net10.0 with `TrimMode=full` and `JsonSerializerIsReflectionEnabledByDefault=true` the v1 anonymous type serialized to the empty object and the guard reported it verbatim — `Expected 13 properties [...] in that order; got 0 []`. Under a trimmed publish the payload could serialize to an empty object, and bytes that back a signature must never be silently empty. All four configurations now emit the golden bytes.
+- **All three producers write their payload field by field with `Utf8JsonWriter`**, from name constants the shape guard reads as well, so the byte order is the statement order in one method and the canonical form is a property of this repository rather than of whichever `System.Text.Json` build a consumer resolves — `Ashlar.Certification.Contracts` and `Ashlar.Certification.State` ship `netstandard2.0`, `net8.0` and `net10.0`, which are three different serializer builds behind one "canonical bytes" claim. **`Utf8JsonWriter` still does the encoding, deliberately**: compact output; the default JavaScript encoder with its uppercase `\uXXXX` (so `+` in a Base64 hash is escaped and `/` and `=` are not); the `"O"`-format UTC timestamp as a **string**, never the writer's own `DateTimeOffset` overload, which emits `+00:00` and trims trailing fractional zeros; the literal `null` at every level rather than an omitted property; and `WriteNumber(double)`, so each target keeps its own decimal form. Choosing one decimal form here would be a different change and would move signatures already written; the corpora restrict doubles to exactly representable values for that reason.
+- **Byte-neutral, and the golden corpus is the acceptance test.** `canonical-payloads.golden.json` is unchanged by this work. All five payloads are byte-identical, byte-length-identical (435 / 276 / 1213 / 391 / 565) and SHA-256-identical: on net8.0 and net10.0 over three consecutive runs each (141 certification tests per run, all green); on the shipped netstandard2.0 asset executed under Mono 6.12; on the .NET 9.0.20 runtime through the packed nupkg bound to `lib/net8.0`; and on net10.0 published `PublishTrimmed`/`TrimMode=partial`, `PublishTrimmed`/`TrimMode=full`, the same with reflection explicitly re-enabled, and `PublishAot` — each self-contained `linux-x64`, each **run**, each with **zero** `IL`/`AOT`/`TRIM` analysis warnings.
+- **The two producers that had no byte pin at all now have one, pinned before they were changed.** `composition-payloads.golden.json` (populated, minimal, escaping, mutant-id ordering) and `transition-entry-hashes.golden.json` (genesis, chained, escaping) sit beside the brick corpus in the same shape with the same regeneration ritual, pinning the payload string, its UTF-8 byte length and its SHA-256 — and, for a transition, the Base64 entry hash a caller can observe through the public API. Both were landed against the unchanged implementations first, so the de-reflection had an oracle rather than an opinion. The composition payload's `BuildPayload` and the transition builder's canonical form became `internal` so the pins can see the bytes; going through a signature or a hash instead would report "it moved" rather than which byte moved it. `scripts/ns20-canonical-bytes-probe.sh` gains the transition entry hashes as a fourth check on the ns2.0 asset, and a new `scripts/portability/trim-aot-canonical-bytes-probe.sh` publishes and runs the four trim/AOT configurations above. Both are repeatable local measurements rather than merge-blocking coverage; no workflow calls either. One thing a corpus cannot pin on its own is which comparer orders a sorted sequence, because ids that read like real mutant ids collate the same way under ordinal and culture rules; the composition corpus gains a case whose ids do not, and `CanonicalPayloadEmitterTests` gains the two the brick lane needs — the mutant-id lists and the `kind`-then-`id` input order.
+- **`EnsureCanonical` stays, and its meaning changes.** It is no longer detecting a serializer degrading under a publish mode — no input to the writer can produce another name sequence — so it is a post-condition on the writer, which is exactly what a mistyped name or a swapped pair of write statements would trip. Two names earn it on their own: camel casing lowercases a leading **run**, so the spellings are `brickId` (not `brickID`) and `ed25519PublicKey` (not `ed25519publicKey`), and a one-character slip in either changes every signature in that lane. The composition payload gains a post-condition of the same shape; the transition builder deliberately does not, because `StateLogVerifier` answers with a refusal code rather than an exception and compares the hash rather than the payload, so there is nowhere for one to land — its byte pin is the check.
+- **Two narrow behaviour changes.** A proposer that supplies the same parameter key twice now raises `CanonicalPayloadException` where the removed intermediate dictionary raised `ArgumentException`: a JSON writer does not reject a duplicate property name, and a payload carrying one describes no single record. The new exception is the one every verification path already catches, so verification refuses (`payload-not-canonical`) instead of throwing into its host. Reaching it needs a custom `IReadOnlyDictionary`; a real `Dictionary` cannot produce it. And `CompositionCertificationRecordSigner.Verify` now catches `CanonicalPayloadException` beside `FormatException`, so that lane refuses where the brick lane already refused instead of throwing.
+- **Still not covered.** Two producers still reach the signed v2 payload through a reflective serializer, and this work changes neither. `SessionEnvironmentInputs` computes `CertificationInput.Hash` by serializing the sandbox spec and the attestation reflectively; `CertificationGate.BuildInputs` (`src/Ashlar.Infrastructure/Certification/CertificationGate.cs:380`) does the same for `request.Witness` and hashes the result into the `witness` input, on the live minting path. `CertificationInput` entries are written **inside** the signed v2 payload, so for either of them a trimmed publish would still produce different signed bytes end to end, for that reason rather than this one. `BrickOutputSerializer.ToCanonicalJson` and `WitnessRunner.CanonicalizeObservation` (witness comparison, not signed) are the same shape. None of them is pinned, guarded or changed here.
 
 ### Fixed
 

ci/cert-gate-assertions.md

@@ -28,6 +28,7 @@ away, can see what they would be turning off.
 | **The schema floor refuses a downgraded record.** A forged record with a rewritten gate name verifies at floor 0 and is refused at floor 2. | `SchemaVersionFloorTests` | A record can claim to have passed gates it never ran: the legacy payload lane leaves `Gate`, `GatesPassed`, `Inputs`, `Proposer`, `Attempts` and `Ed25519PublicKey` outside the signed bytes. |
 | **The dev key is loud.** A signer falling back to the committed HMAC constant warns, and never logs the key itself. | `CertificationRecordSignerDevKeyTests` | Production admissions run on a key anyone with the source can forge, with nothing on the record saying so. |
 | **v1/v2 record bytes are frozen.** Both lanes are byte-pinned against a checked-in golden corpus (`canonical-payloads.golden.json`), which `scripts/ns20-canonical-bytes-probe.sh` (the netstandard2.0 asset under Mono) and `scripts/portability/net9-probe.sh` (the net8.0 asset on the 9.0 runtime) also check against, and a payload that does not carry its lane’s exact property-name sequence is refused rather than signed. | `TrustLoopRecordSchemaTests`, `CanonicalPayloadGoldenTests`, `CanonicalPayloadGuardTests` | Every signature ever written becomes unverifiable, silently. |
+| **The other two canonical-bytes producers are frozen too.** The composition admission payload and the certified transition entry hash are byte-pinned against `composition-payloads.golden.json` and `transition-entry-hashes.golden.json`, and written field by field rather than serialized, so a trimmed or ahead-of-time publish emits the same bytes — measured by `scripts/portability/trim-aot-canonical-bytes-probe.sh`, which publishes and runs four such configurations. | `CompositionCanonicalPayloadGoldenTests`, `CertifiedTransitionEntryHashGoldenTests`, `CanonicalPayloadEmitterTests` | Every composition certificate and every attested state log stops verifying, silently, on a host that publishes differently from CI. |
 | **Every target reaches the same verdict for the same record.** A record carrying an Ed25519 signature that does not verify is refused under every options instance (Legacy included), and the netstandard2.0 asset — which cannot evaluate the signature — refuses the same corpus records (`ed25519-signature-unverifiable`) instead of trusting them; HMAC-only records stay trusted on both. | `VerifierParityTests` (net8.0 side; `scripts/ns20-canonical-bytes-probe.sh` measures the netstandard2.0 side under Mono against the same corpus) | A downlevel consumer trusts a record every other target refuses, and a certificate's verdict depends on which framework happened to load the verifier. |
 | **An unknown schema version is an error, not a guess.** Only null (v1) and 2 (v2) select a canonical payload lane; any other version is refused at mint time and on every verification path (`schema-version-unknown` from the trust verifier), and the null/2 bytes are unchanged against the golden corpus. | `UnknownSchemaVersionTests`, `CanonicalPayloadGoldenTests` | A record stamped with a version no code has ever minted is serialized in a shape chosen by guesswork and clears every schema floor at or below its number. |
 | **No eighth unbounded appender.** Every production `File.AppendAllText` / `AppendAllLines` / `AppendText` sits in a frozen allowlist of the seven that exist; a stale allowlist row fails too, so the inventory can only shrink honestly. | `AppendOnlyWriterConventionTests` | An appender on a path that never rotates grows until the disk does not — weeks later, on an unattended node, long after the change that caused it. `CLOSING-PLAN.md` Phase 5 bounds these at the write path; this stops the count going up in the meantime. |

docs/specs/SPEC-006-keys-and-signing.md

@@ -60,8 +60,12 @@ deliberately NOT the same bytes and must not be converged:
 
 - Operator artifacts (`src/Ashlar.Manifest/Signing/CanonicalJson.cs`) recursively ordinal-sort
   keys at every depth, use default (Pascal) casing, and omit nulls.
-- Certification records (`src/Ashlar.Certification.Contracts/CertificationRecordSigning.cs`, `BuildPayload`) serialize a positional
-  DTO in declaration order, camelCase, writing nulls.
+- Certification records (`src/Ashlar.Certification.Contracts/CertificationRecordSigning.cs`, `BuildPayload`) write the payload
+  field by field with `Utf8JsonWriter`, in a fixed order, camelCase names, writing nulls. The
+  order is the order of the write statements and the names are constants the shape guard reads
+  too, so the canonical form is a property of that file rather than of the serializer a
+  consumer resolves — which is what lets the same bytes come out of a trimmed or
+  ahead-of-time publish. `canonical-payloads.golden.json` pins them.
 
 *Corrected 2026-08-27.* This paragraph previously asserted that certification records
 already established the ordinal-sorted convention and that v1 "REUSES that machinery". That