From 6939db9b16ea08cb41703fa0d97e4fb388e9c930 Mon Sep 17 00:00:00 2001 From: IanFrelinger Date: Thu, 10 Sep 2026 18:43:28 -0400 Subject: [PATCH 01/11] refactor(certification): write the canonical signing payload instead of serializing it The canonical payload is the message every certification signature -- HMAC and Ed25519, v1 and v2 -- is computed over, so the bytes must be produced the same way in every publish mode a consumer can choose. They were produced by reflection-based `JsonSerializer.Serialize`: the v1 lane over an anonymous type, the v2 lane over a private positional record. Reflection-based serialization does not survive trimming or ahead-of-time publishing. Measured in the devtest container across the publish matrix on a self-contained linux-x64 publish, the same source produced correct bytes, a throw, or an empty object depending only on the target framework and the trim settings: under `PublishAot` with defaults and under `PublishTrimmed` with either `TrimMode` both lanes throw `InvalidOperationException` (reflection-based serialization disabled); on net10.0 with `TrimMode=full` and `JsonSerializerIsReflectionEnabledByDefault=true` the v1 anonymous type serialized to `{}` while the named v2 record came out correct; on net8.0 with `TrimMode=full` and reflection re-enabled both lanes throw `NotSupportedException` about constructor parameter names ILLink had trimmed. Under a trimmed publish the payload could serialize to an empty object, and bytes that back a signature must never be silently empty. `BuildPayload` now writes the payload field by field with `Utf8JsonWriter`. The byte order is the statement order in one method and the names are constants read by both the writer and the shape guard, so the canonical form is a property of this file rather than of whichever `System.Text.Json` build a consumer resolves -- the package ships netstandard2.0, net8.0 and net10.0, which are three different serializer builds behind one "canonical bytes" claim. `Utf8JsonWriter` still does the encoding, deliberately: its defaults are compact output and the default JavaScript encoder, and `WriteNumber(double)` keeps whatever decimal form the target produces, so each target's existing behaviour is reproduced rather than replaced. Choosing one decimal form would be a different change and would move signatures already written. BYTE-NEUTRAL. `canonical-payloads.golden.json` is unchanged and is the acceptance test: all five cases -- both lanes populated, both lanes minimal, and the escaping case -- are byte-identical, byte-length-identical and SHA-256-identical to the corpus. #584 pinned those bytes and #585 fixed the lane selection, which is what makes this change verifiable by construction rather than by inspection. `EnsureCanonical` is unchanged and still called on the writer's output. It is no longer detecting a serializer degrading under a publish mode -- no input to the writer can produce another name sequence -- so it is now a post-condition on the writer, which is exactly what a mistyped name or a swapped pair of write statements would trip. Two names are worth the guard on their own: camel casing lowercases a leading run, so the correct spellings are `brickId` and `ed25519PublicKey`, and a one-character slip in either changes every signature in that lane. Two behaviour notes. Nulls, sort orders, structural exclusions, the `"O"`-format UTC timestamp string and the verbatim ordinal-sorted proposer parameter keys are all preserved exactly. A proposer that supplies the same parameter key twice now raises `CanonicalPayloadException` rather than the `ArgumentException` the removed intermediate dictionary raised: `Utf8JsonWriter` does not reject a duplicate property name, and a payload carrying one describes no single record. The new vehicle is the one `VerifySignature` already catches, so verification refuses instead of throwing into its host -- which is the documented contract everywhere else on that path. Reaching it needs a custom `IReadOnlyDictionary`; a real `Dictionary` cannot produce it. Co-Authored-By: Claude Opus 5 --- .../CanonicalPayloadException.cs | 23 +- .../CertificationRecordSigning.cs | 386 ++++++++++++------ .../CertificationTrustVerifier.cs | 11 +- 3 files changed, 290 insertions(+), 130 deletions(-) diff --git a/src/Ashlar.Certification.Contracts/CanonicalPayloadException.cs b/src/Ashlar.Certification.Contracts/CanonicalPayloadException.cs index ba46b535f..e75a92e52 100644 --- a/src/Ashlar.Certification.Contracts/CanonicalPayloadException.cs +++ b/src/Ashlar.Certification.Contracts/CanonicalPayloadException.cs @@ -1,15 +1,22 @@ namespace Ashlar.Certification.Contracts; /// -/// Thrown when the canonical signing payload does not have the exact shape its lane declares. +/// Thrown when the canonical signing payload does not have the exact shape its lane declares, +/// or cannot be written at all. /// -/// The canonical payload is built by reflection-based System.Text.Json, which is not a -/// guarantee: reflection-based serialization does not survive trimming or ahead-of-time -/// publishing, and under those publish modes the payload can come out as the empty object -/// {} or missing properties, with no exception and no warning. Those bytes are the -/// message every HMAC and Ed25519 certification signature is computed over, so a payload that -/// is not the declared shape does not describe the record it claims to cover. Minting one is -/// refused; the alternative is a certificate whose bytes nobody can vouch for. +/// Those bytes are the message every HMAC and Ed25519 certification signature is computed over, +/// so a payload that is not the declared shape does not describe the record it claims to cover. +/// Minting one is refused; the alternative is a certificate whose bytes nobody can vouch for. +/// The payload is written field by field rather than serialized from an object graph, precisely +/// so that its shape is decided here: reflection-based serialization does not survive trimming +/// or ahead-of-time publishing, and under a trimmed publish the payload could serialize to an +/// empty object with no exception and no warning. What remains is a post-condition on the +/// writer, which is what an editing mistake in it would trip. +/// +/// +/// Also thrown when the payload cannot be written as a single well-formed record at all — a +/// proposer supplying the same parameter key twice, whose keys are emitted verbatim and would +/// therefore produce a duplicate property name. /// /// /// Also thrown when the record's schema version selects no lane at all diff --git a/src/Ashlar.Certification.Contracts/CertificationRecordSigning.cs b/src/Ashlar.Certification.Contracts/CertificationRecordSigning.cs index 18f77a856..4fd3085e2 100644 --- a/src/Ashlar.Certification.Contracts/CertificationRecordSigning.cs +++ b/src/Ashlar.Certification.Contracts/CertificationRecordSigning.cs @@ -96,8 +96,6 @@ private static bool FixedTimeEquals(byte[] left, byte[] right) #endif } - private static readonly JsonSerializerOptions PayloadOptions = new() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }; - /// /// Whether selects a canonical payload lane: null is the /// legacy v1 lane, is v2, and @@ -123,20 +121,19 @@ public static bool IsKnownSchemaVersion(int? schemaVersion) => /// /// Record to serialize. /// - /// The serializer did not produce the lane's declared shape, or the record's schema version - /// selects no lane at all. Serialization here is reflection-based, which does not survive - /// trimming or ahead-of-time publishing, so the payload can silently come out empty or - /// short; and a version this code has never seen would otherwise be serialized under a - /// shape chosen by guesswork. These bytes back every signature, so in either case they are - /// refused rather than signed. + /// The emitter did not produce the lane's declared shape, the record's schema version + /// selects no lane at all, or a proposer parameter key repeats so the payload would carry a + /// duplicate property name and describe no single record. A version this code has never + /// seen would otherwise be written under a shape chosen by guesswork. These bytes back + /// every signature, so in each case they are refused rather than signed. /// public static string BuildPayload(CertificationRecordData record) { if (record.SchemaVersion is null) - return EnsureCanonical(BuildLegacyPayload(record), versioned: false); + return EnsureCanonical(WritePayload(record, schemaVersion: null), versioned: false); if (record.SchemaVersion.Value == CertificationRecordData.TrustLoopSchemaVersion) - return EnsureCanonical(BuildVersionedPayload(record, record.SchemaVersion.Value), versioned: true); + return EnsureCanonical(WritePayload(record, record.SchemaVersion.Value), versioned: true); // An unknown version selects no lane. Serializing it under the v2 shape would emit the // version verbatim inside bytes whose meaning this code cannot know, and a record so @@ -149,104 +146,291 @@ public static string BuildPayload(CertificationRecordData record) + "so no signature can be computed or checked over it."); } - private static string BuildVersionedPayload(CertificationRecordData record, int schemaVersion) + // The largest payload in the golden corpus is 1213 bytes. A starting size, not a limit. + private const int PayloadBufferHint = 2048; + + // The canonical payload is WRITTEN, not serialized from an object graph. Reflection-based + // serialization does not survive trimming or ahead-of-time publishing: under a trimmed + // publish the payload could serialize to an empty object, and bytes that back a signature + // must never be silently empty. Written this way the byte order is the statement order + // below and the names are the constants above, so the canonical form is a property of this + // file rather than of whichever System.Text.Json build a consumer resolves — this package + // ships netstandard2.0, net8.0 and net10.0, which are three different serializer builds + // behind one "canonical bytes" claim. + // + // Utf8JsonWriter still does the encoding, deliberately: its defaults are compact output and + // the default JavaScript encoder, and WriteNumber(double) keeps whatever decimal form the + // target produces. This change is about how the bytes are ORDERED and NAMED and must not + // move one of them; canonical-payloads.golden.json is what says it did not. + private static string WritePayload(CertificationRecordData record, int? schemaVersion) + { + using var buffer = new MemoryStream(PayloadBufferHint); + using (var writer = new Utf8JsonWriter(buffer)) + { + writer.WriteStartObject(); + + // v2 is v1's names in the same order with schemaVersion prepended and six appended, + // so the two lanes are one method under two brackets rather than two transcriptions + // that could drift apart in the thirteen names they share. + if (schemaVersion is not null) + writer.WriteNumber(Names.SchemaVersion, schemaVersion.Value); + + WriteStringOrNull(writer, Names.Status, record.Status); + WriteStringOrNull(writer, Names.Stage, record.Stage); + writer.WriteBoolean(Names.Admitted, record.Admitted); + writer.WriteBoolean(Names.Signed, record.Signed); + + // The round-trip form of the UTC DateTime, written as a STRING. Utf8JsonWriter's own + // DateTimeOffset overload emits "+00:00" and trims trailing fractional zeros, which + // is a different message for the same instant. + WriteStringOrNull(writer, Names.Timestamp, record.Timestamp.UtcDateTime.ToString("O")); + WriteStringOrNull(writer, Names.BrickId, record.BrickId); + WriteStringOrNull(writer, Names.ContentHash, record.ContentHash); + WriteNumberOrNull(writer, Names.EscapeRate, record.EscapeRate); + WriteNumberOrNull(writer, Names.TotalMutants, record.TotalMutants); + WriteNumberOrNull(writer, Names.SurvivingMutants, record.SurvivingMutants); + WriteOrdinalSorted(writer, Names.KilledMutants, record.KilledMutants); + WriteOrdinalSorted(writer, Names.SurvivingMutantIds, record.SurvivingMutantIds); + WriteStringOrNull(writer, Names.Reason, record.Reason); + + if (schemaVersion is not null) + { + WriteStringOrNull(writer, Names.Gate, record.Gate); + + // Gate and attempt order is semantic and preserved; inputs and mutant ids are + // sorted, because their caller order carries nothing and would otherwise be the + // thing that decides the bytes. + writer.WriteStartArray(Names.GatesPassed); + foreach (var gatePass in record.GatesPassed) + { + writer.WriteStartObject(); + WriteStringOrNull(writer, Names.Name, gatePass.Name); + WriteStringOrNull(writer, Names.Version, gatePass.Version); + WriteStringOrNull(writer, Names.Configuration, gatePass.Configuration); + writer.WriteEndObject(); + } + + writer.WriteEndArray(); + + writer.WriteStartArray(Names.Inputs); + foreach (var input in record.Inputs + .OrderBy(i => i.Kind, StringComparer.Ordinal) + .ThenBy(i => i.Id, StringComparer.Ordinal)) + { + writer.WriteStartObject(); + WriteStringOrNull(writer, Names.Kind, input.Kind); + WriteStringOrNull(writer, Names.Id, input.Id); + WriteStringOrNull(writer, Names.Hash, input.Hash); + writer.WriteEndObject(); + } + + writer.WriteEndArray(); + + WriteProposer(writer, record.Proposer); + + writer.WriteStartArray(Names.Attempts); + foreach (var attempt in record.Attempts) + { + writer.WriteStartObject(); + writer.WriteNumber(Names.Index, attempt.Index); + WriteStringOrNull(writer, Names.Outcome, attempt.Outcome); + WriteStringOrNull(writer, Names.FailureCategory, attempt.FailureCategory); + WriteNumberOrNull(writer, Names.DurationSeconds, attempt.DurationSeconds); + writer.WriteEndObject(); + } + + writer.WriteEndArray(); + + WriteStringOrNull(writer, Names.Ed25519PublicKey, record.Ed25519PublicKey); + } + + writer.WriteEndObject(); + } + + return Encoding.UTF8.GetString(buffer.ToArray()); + } + + // proposer.parameters is the one place in the payload whose property names are supplied by + // the caller: no key policy is configured, so keys are emitted verbatim in ordinal order. + // They are written straight from the sorted sequence — the intermediate dictionary the + // previous code built to carry them relied on Dictionary<,> preserving insertion + // order, which is a property of that type rather than a statement this payload makes. + private static void WriteProposer(Utf8JsonWriter writer, CertificationProposer? proposer) + { + if (proposer is null) + { + // The literal null, never an empty object: a record with no proposer and a record + // with an empty one are different records and must not share a signature. + writer.WriteNull(Names.Proposer); + return; + } + + writer.WriteStartObject(Names.Proposer); + WriteStringOrNull(writer, Names.Identity, proposer.Identity); + + writer.WriteStartObject(Names.Parameters); + string? previousKey = null; + foreach (var parameter in proposer.Parameters.OrderBy(p => p.Key, StringComparer.Ordinal)) + { + // The sort is ordinal, so a repeated key arrives next to its twin. Utf8JsonWriter + // does not reject a duplicate property name, and a payload carrying one describes no + // single record, so it is refused in the same vehicle every other unusable payload + // uses. The key itself is not named: this message reaches logs and refusal text, and + // these keys are caller data rather than shape. + if (string.Equals(previousKey, parameter.Key, StringComparison.Ordinal)) + { + throw new CanonicalPayloadException( + "The v2 canonical certification payload cannot be written: the proposer supplied " + + "the same parameter key more than once, so the payload would carry a duplicate " + + "property name and describe no single record."); + } + + previousKey = parameter.Key; + WriteStringOrNull(writer, parameter.Key, parameter.Value); + } + + writer.WriteEndObject(); + + WriteStringOrNull(writer, Names.Seed, proposer.Seed); + writer.WriteEndObject(); + } + + // Every helper below writes the literal null rather than omitting the property, at every + // level. Nothing here is conditional on a value, which is what keeps the name sequence an + // invariant of the lane rather than of the record — and therefore what lets EnsureCanonical + // define a degenerate payload as a shape rather than as a size. + private static void WriteStringOrNull(Utf8JsonWriter writer, string name, string? value) { - var clone = new VersionedPayload( - schemaVersion, - record.Status, - record.Stage, - record.Admitted, - record.Signed, - record.Timestamp.UtcDateTime.ToString("O"), - record.BrickId, - record.ContentHash, - record.EscapeRate, - record.TotalMutants, - record.SurvivingMutants, - record.KilledMutants.OrderBy(x => x, StringComparer.Ordinal).ToArray(), - record.SurvivingMutantIds.OrderBy(x => x, StringComparer.Ordinal).ToArray(), - record.Reason, - record.Gate, - record.GatesPassed.Select(g => new GatePassPayload(g.Name, g.Version, g.Configuration)).ToArray(), - record.Inputs - .OrderBy(i => i.Kind, StringComparer.Ordinal) - .ThenBy(i => i.Id, StringComparer.Ordinal) - .Select(i => new InputPayload(i.Kind, i.Id, i.Hash)) - .ToArray(), - record.Proposer is null - ? null - : new ProposerPayload( - record.Proposer.Identity, - record.Proposer.Parameters - .OrderBy(p => p.Key, StringComparer.Ordinal) - .ToDictionary(p => p.Key, p => p.Value), - record.Proposer.Seed), - record.Attempts.Select(a => new AttemptPayload(a.Index, a.Outcome, a.FailureCategory, a.DurationSeconds)).ToArray(), - record.Ed25519PublicKey); - return JsonSerializer.Serialize(clone, PayloadOptions); + if (value is null) + writer.WriteNull(name); + else + writer.WriteString(name, value); } - private static string BuildLegacyPayload(CertificationRecordData record) + private static void WriteNumberOrNull(Utf8JsonWriter writer, string name, int? value) { - var clone = new + if (value is null) + writer.WriteNull(name); + else + writer.WriteNumber(name, value.Value); + } + + // WriteNumber(double) rather than a formatted string. The decimal form of an arbitrary + // double is not identical across the three targets this package ships for, and reproducing + // each target's own form is what keeps this change byte-neutral; choosing one form here + // would be a different change, and would move signatures already written. The golden corpus + // restricts doubles to exactly representable values for the same reason. + private static void WriteNumberOrNull(Utf8JsonWriter writer, string name, double? value) + { + if (value is null) + writer.WriteNull(name); + else + writer.WriteNumber(name, value.Value); + } + + private static void WriteOrdinalSorted(Utf8JsonWriter writer, string name, IReadOnlyList values) + { + writer.WriteStartArray(name); + foreach (string? value in values.OrderBy(x => x, StringComparer.Ordinal)) { - record.Status, - record.Stage, - record.Admitted, - record.Signed, - Timestamp = record.Timestamp.UtcDateTime.ToString("O"), - record.BrickId, - record.ContentHash, - record.EscapeRate, - record.TotalMutants, - record.SurvivingMutants, - KilledMutants = record.KilledMutants.OrderBy(x => x, StringComparer.Ordinal).ToArray(), - SurvivingMutantIds = record.SurvivingMutantIds.OrderBy(x => x, StringComparer.Ordinal).ToArray(), - record.Reason - }; - return JsonSerializer.Serialize(clone, PayloadOptions); + if (value is null) + writer.WriteNullValue(); + else + writer.WriteStringValue(value); + } + + writer.WriteEndArray(); + } + + // Every property name in the payload, spelled once and read twice: the emitter writes them + // and the shape guard below checks them, so the two cannot disagree about a spelling. Two of + // these are not what inspection suggests — camel casing lowercases a leading RUN, which is + // why it is "brickId" and not "brickID", and "ed25519PublicKey" and not "ed25519publicKey" — + // and a one-character difference in either changes every signature in that lane. + private static class Names + { + internal const string SchemaVersion = "schemaVersion"; + internal const string Status = "status"; + internal const string Stage = "stage"; + internal const string Admitted = "admitted"; + internal const string Signed = "signed"; + internal const string Timestamp = "timestamp"; + internal const string BrickId = "brickId"; + internal const string ContentHash = "contentHash"; + internal const string EscapeRate = "escapeRate"; + internal const string TotalMutants = "totalMutants"; + internal const string SurvivingMutants = "survivingMutants"; + internal const string KilledMutants = "killedMutants"; + internal const string SurvivingMutantIds = "survivingMutantIds"; + internal const string Reason = "reason"; + internal const string Gate = "gate"; + internal const string GatesPassed = "gatesPassed"; + internal const string Inputs = "inputs"; + internal const string Proposer = "proposer"; + internal const string Attempts = "attempts"; + internal const string Ed25519PublicKey = "ed25519PublicKey"; + internal const string Name = "name"; + internal const string Version = "version"; + internal const string Configuration = "configuration"; + internal const string Kind = "kind"; + internal const string Id = "id"; + internal const string Hash = "hash"; + internal const string Identity = "identity"; + internal const string Parameters = "parameters"; + internal const string Seed = "seed"; + internal const string Index = "index"; + internal const string Outcome = "outcome"; + internal const string FailureCategory = "failureCategory"; + internal const string DurationSeconds = "durationSeconds"; } // The property-NAME sequence each lane must produce. This is what makes "degenerate" - // definable without guessing at size or emptiness: PayloadOptions carries no - // JsonIgnoreCondition, so every declared property is emitted for every record whatever its - // values, and a minimal legitimate record still carries every one of these names with - // nulls. The sequence therefore depends on the payload TYPE and not on any field value — - // there is no legitimate record it can reject — and it catches in one check everything - // reflection-based serialization can lose when it is trimmed away or published - // ahead-of-time: an empty object, missing properties, and a reordering, which would - // silently invalidate every signature ever written. + // definable without guessing at size or emptiness: nothing is ever omitted, so every + // declared property is emitted for every record whatever its values, and a minimal + // legitimate record still carries every one of these names with nulls. The sequence + // therefore depends on the lane and not on any field value — there is no legitimate record + // it can reject — and it catches in one check an empty object, a missing property, and a + // reordering, which loses no information but changes the bytes and would silently + // invalidate every signature ever written. + // + // The emitter above leaves the serializer no room to decide any of that, which makes this a + // post-condition an editing mistake trips rather than a deployment fault it detects. It is + // kept for exactly that reason: it is the one thing standing between a mistyped name or a + // swapped pair of statements and a silently re-shaped signed payload. private static readonly string[] LegacyPayloadNames = { - "status", "stage", "admitted", "signed", "timestamp", "brickId", "contentHash", - "escapeRate", "totalMutants", "survivingMutants", "killedMutants", "survivingMutantIds", - "reason" + Names.Status, Names.Stage, Names.Admitted, Names.Signed, Names.Timestamp, Names.BrickId, + Names.ContentHash, Names.EscapeRate, Names.TotalMutants, Names.SurvivingMutants, + Names.KilledMutants, Names.SurvivingMutantIds, Names.Reason }; private static readonly string[] VersionedPayloadNames = { - "schemaVersion", "status", "stage", "admitted", "signed", "timestamp", "brickId", - "contentHash", "escapeRate", "totalMutants", "survivingMutants", "killedMutants", - "survivingMutantIds", "reason", "gate", "gatesPassed", "inputs", "proposer", "attempts", - "ed25519PublicKey" + Names.SchemaVersion, Names.Status, Names.Stage, Names.Admitted, Names.Signed, + Names.Timestamp, Names.BrickId, Names.ContentHash, Names.EscapeRate, Names.TotalMutants, + Names.SurvivingMutants, Names.KilledMutants, Names.SurvivingMutantIds, Names.Reason, + Names.Gate, Names.GatesPassed, Names.Inputs, Names.Proposer, Names.Attempts, + Names.Ed25519PublicKey }; - private static readonly string[] GatePassNames = { "name", "version", "configuration" }; + private static readonly string[] GatePassNames = { Names.Name, Names.Version, Names.Configuration }; - private static readonly string[] InputNames = { "kind", "id", "hash" }; + private static readonly string[] InputNames = { Names.Kind, Names.Id, Names.Hash }; - private static readonly string[] ProposerNames = { "identity", "parameters", "seed" }; + private static readonly string[] ProposerNames = { Names.Identity, Names.Parameters, Names.Seed }; - private static readonly string[] AttemptNames = { "index", "outcome", "failureCategory", "durationSeconds" }; + private static readonly string[] AttemptNames = { Names.Index, Names.Outcome, Names.FailureCategory, Names.DurationSeconds }; /// /// Returns when it carries the exact property-name sequence its - /// lane declares, and throws otherwise. Re-parsing with is - /// deliberate: the parser is reflection-free, so it still works in exactly the publish - /// configurations that break the serializer that produced the payload. - /// Internal rather than private so the guard can be exercised against payloads this - /// process cannot make the serializer emit — the publish modes that produce them are not - /// reproducible inside a test host. + /// lane declares, and throws otherwise. No input to the emitter can produce another + /// sequence, so this is a post-condition on the emitter rather than a check on the runtime: + /// it asserts that the bytes about to back a signature are the shape this file says they + /// are, which is what a mistyped name or a swapped pair of write statements would break. + /// Re-parsing with keeps the check reflection-free, so it holds + /// in every publish mode the emitter itself holds in. + /// Internal rather than private so it can be exercised directly against payloads no input + /// to the emitter can produce. /// internal static string EnsureCanonical(string payload, bool versioned) { @@ -345,36 +529,6 @@ private static JsonElement RequireArray(JsonElement element, string lane, string return element; } - private sealed record VersionedPayload( - int SchemaVersion, - string Status, - string Stage, - bool Admitted, - bool Signed, - string Timestamp, - string BrickId, - string? ContentHash, - double? EscapeRate, - int? TotalMutants, - int? SurvivingMutants, - string[] KilledMutants, - string[] SurvivingMutantIds, - string? Reason, - string? Gate, - GatePassPayload[] GatesPassed, - InputPayload[] Inputs, - ProposerPayload? Proposer, - AttemptPayload[] Attempts, - string? Ed25519PublicKey); - - private sealed record GatePassPayload(string Name, string? Version, string? Configuration); - - private sealed record InputPayload(string Kind, string Id, string Hash); - - private sealed record ProposerPayload(string Identity, Dictionary Parameters, string? Seed); - - private sealed record AttemptPayload(int Index, string Outcome, string? FailureCategory, double? DurationSeconds); - private static string ResolveKey(string? hmacKey) { if (!string.IsNullOrWhiteSpace(hmacKey)) diff --git a/src/Ashlar.Certification.Contracts/CertificationTrustVerifier.cs b/src/Ashlar.Certification.Contracts/CertificationTrustVerifier.cs index ac0bf3d8d..1892e6948 100644 --- a/src/Ashlar.Certification.Contracts/CertificationTrustVerifier.cs +++ b/src/Ashlar.Certification.Contracts/CertificationTrustVerifier.cs @@ -45,9 +45,9 @@ public static CertificationTrustResult Verify( // The floor says "at least this new"; it does not say "a version this verifier knows". // BuildPayload selects its lane on the version, and a version it has never seen selects // no lane, so there are no bytes to compare any signature against. Refused with its own - // code rather than left to surface as payload-not-canonical, because that code is - // documented as the serializer degrading under trimming or ahead-of-time publishing — a - // deployment fault — and an unknown version is a record fault; conflating them would + // code rather than left to surface as payload-not-canonical, because that code means the + // bytes this build wrote are not the shape it declares — a fault in this code — and an + // unknown version is a record fault; conflating them would // point an operator at the wrong thing. An unknown schema version is an error, not a // guess. Floor first, so an explicit version below the floor still reports the floor. if (!CertificationRecordSigning.IsKnownSchemaVersion(record.SchemaVersion)) @@ -66,9 +66,8 @@ public static CertificationTrustResult Verify( return Untrusted("content-hash-missing", "Certification record has no content hash."); // Built BEFORE the signature is compared, and unconditionally. Both sides of a comparison - // recompute the payload from the same serializer, so the comparison cannot establish - // anything about bytes whose shape has not been established first. Establish the shape, - // then compare. + // recompute the payload the same way, so the comparison cannot establish anything about + // bytes whose shape has not been established first. Establish the shape, then compare. try { CertificationRecordSigning.BuildPayload(record); From 260d2deed9facf6feeedb10090b50e464de433d1 Mon Sep 17 00:00:00 2001 From: IanFrelinger Date: Thu, 10 Sep 2026 18:50:52 -0400 Subject: [PATCH 02/11] test(certification): pin the composition certification payload bytes `CompositionCertificationRecordSigner` produces the message every composition admission signature is computed over -- `CompositionCertificationGate` mints one for every admitted composition, and `CertifiedCompositionRegistry` verifies it -- and nothing pinned those bytes. The brick lane has had a byte-for-byte corpus since #584; this lane could change shape, order or spelling and the only thing that would notice is a stored certificate failing to verify some time after the change shipped. `composition-payloads.golden.json` sits beside the brick corpus in the same shape and with the same regeneration ritual: a populated record, a minimal one, and an escaping case, each pinning the payload string, its UTF-8 byte length and its SHA-256 separately, so a change a string comparison could normalise away still fails. Doubles are restricted to exactly representable values (0.5 and 0.25) for the reason the brick corpus states. The populated fixture supplies `signature` and `gate` as sentinels, so the structural exclusion is asserted rather than assumed. `BuildPayload` becomes internal so the pin can observe the bytes themselves. Going through a signature instead would report "the signature moved" rather than which byte did, and `Ashlar.Infrastructure` already grants `InternalsVisibleTo` to this test assembly. Pinned against the current implementation, unchanged: all three cases pass as-is. That is the point of landing it first -- the corpus records what this signer emits today, so a later change to how it emits has an oracle rather than an opinion. Co-Authored-By: Claude Opus 5 --- .../CompositionCertificationRecordSigner.cs | 6 +- .../Ashlar.Tests.Infrastructure.csproj | 4 + .../CompositionCanonicalPayloadGoldenTests.cs | 143 ++++++++++++++++++ .../composition-payloads.golden.json | 73 +++++++++ 4 files changed, 225 insertions(+), 1 deletion(-) create mode 100644 src/Ashlar.Tests.Infrastructure/Tests/Certification/CompositionCanonicalPayloadGoldenTests.cs create mode 100644 src/Ashlar.Tests.Infrastructure/Tests/Certification/composition-payloads.golden.json diff --git a/src/Ashlar.Infrastructure/Certification/Composition/CompositionCertificationRecordSigner.cs b/src/Ashlar.Infrastructure/Certification/Composition/CompositionCertificationRecordSigner.cs index 8cb074a06..34f906546 100644 --- a/src/Ashlar.Infrastructure/Certification/Composition/CompositionCertificationRecordSigner.cs +++ b/src/Ashlar.Infrastructure/Certification/Composition/CompositionCertificationRecordSigner.cs @@ -78,7 +78,11 @@ public bool Verify(CompositionCertificationRecord record) } } - private static string BuildPayload(CompositionCertificationRecord record) + // Internal rather than private so CompositionCanonicalPayloadGoldenTests can pin these + // bytes directly. They are the message every composition admission signature is computed + // over, and a pin that could only observe them through a signature would report "the + // signature moved" rather than which byte did. + internal static string BuildPayload(CompositionCertificationRecord record) { var clone = new { diff --git a/src/Ashlar.Tests.Infrastructure/Ashlar.Tests.Infrastructure.csproj b/src/Ashlar.Tests.Infrastructure/Ashlar.Tests.Infrastructure.csproj index 00ed62b23..952665707 100644 --- a/src/Ashlar.Tests.Infrastructure/Ashlar.Tests.Infrastructure.csproj +++ b/src/Ashlar.Tests.Infrastructure/Ashlar.Tests.Infrastructure.csproj @@ -17,6 +17,10 @@ it from the test output, scripts/ns20-canonical-bytes-probe.sh reads the same file from the repo, so the two cannot drift apart and claim an equality neither checked. --> + + diff --git a/src/Ashlar.Tests.Infrastructure/Tests/Certification/CompositionCanonicalPayloadGoldenTests.cs b/src/Ashlar.Tests.Infrastructure/Tests/Certification/CompositionCanonicalPayloadGoldenTests.cs new file mode 100644 index 000000000..19a68384b --- /dev/null +++ b/src/Ashlar.Tests.Infrastructure/Tests/Certification/CompositionCanonicalPayloadGoldenTests.cs @@ -0,0 +1,143 @@ +using System.Security.Cryptography; +using System.Text; +using System.Text.Json; +using Ashlar.Core.Application.Certification.Models; +using Ashlar.Infrastructure.Certification.Composition; +using FluentAssertions; +using Xunit; + +namespace Ashlar.Tests.Infrastructure.Tests.Certification; + +/// +/// Byte-level pin on the canonical signing payload for composition certification records. +/// +/// These bytes are the message every composition admission signature is computed over, and +/// CompositionCertificationGate mints one for every admitted composition. Until this +/// suite existed the brick lane was pinned byte-for-byte and this lane was pinned by nothing: +/// its payload could change shape, order or spelling and the only thing that would notice is a +/// stored certificate failing to verify long after the change shipped. +/// +/// +/// The corpus lives in composition-payloads.golden.json beside the brick one, in the +/// same shape and with the same regeneration ritual, so a reader who knows one knows both. +/// Doubles are restricted to exactly representable values there for the reason that file +/// states. +/// +/// +[Trait("Category", "Certification")] +public sealed class CompositionCanonicalPayloadGoldenTests +{ + private const string GoldenFileName = "composition-payloads.golden.json"; + + private static readonly IReadOnlyDictionary Corpus = LoadCorpus(); + + public static TheoryData GoldenCaseNames + { + get + { + var data = new TheoryData(); + foreach (var name in Corpus.Keys.OrderBy(n => n, StringComparer.Ordinal)) + data.Add(name); + return data; + } + } + + [Theory] + [MemberData(nameof(GoldenCaseNames))] + public void CompositionCanonicalPayload_IsByteIdenticalToGolden(string caseName) + { + var golden = Corpus[caseName]; + + var payload = CompositionCertificationRecordSigner.BuildPayload(golden.Record); + + // The string comparison comes first on purpose: it is the assertion that prints a + // readable diff. The length and digest below turn "readable" into "byte-exact", so a + // change that a string comparison could normalise away still fails. + payload.Should().Be( + golden.Payload, + "the canonical payload for '{0}' backs every composition signature already written " + + "over it; changing these bytes invalidates them", + caseName); + + var bytes = Encoding.UTF8.GetBytes(payload); + bytes.Length.Should().Be(golden.PayloadByteLength, "the signed message is bytes, not characters"); + Sha256Hex(bytes).Should().Be( + golden.PayloadSha256, + "SHA-256 of the UTF-8 canonical composition payload for '{0}'. Actual payload was: {1}", + caseName, + payload); + } + + [Fact] + public void GoldenCorpus_CoversAPopulatedAndAMinimalRecord() + { + // A theory over an empty or half-empty corpus passes without asserting anything, which + // is the same failure mode as having no golden test at all. Assert the corpus itself. + Corpus.Should().NotBeEmpty(); + Corpus.Values.Should().Contain(c => c.Record.CompositionEscapeRate != null, "a populated record is the shape the gate mints"); + Corpus.Values.Should().Contain(c => c.Record.CompositionEscapeRate == null, "a refusal record carries nulls and must still carry every name"); + Corpus.Values.Should().OnlyContain( + c => c.Payload.Length > 2, + "an empty object is never a legitimate canonical payload — every declared property is emitted whatever its value"); + } + + [Fact] + public void Payload_ExcludesTheSignatureAndTheGate() + { + // Stated as its own assertion rather than left implicit in the byte comparison: the + // populated fixture supplies both fields as sentinels, so if either ever entered the + // payload the record would sign a value it does not cover today and every stored + // signature would stop verifying. + var payload = CompositionCertificationRecordSigner.BuildPayload(Corpus["composition-populated"].Record); + + payload.Should().NotContain("MUST-NOT-APPEAR"); + } + + [Fact] + public void Sign_ComputesTheHmacOverExactlyTheGoldenBytes() + { + // Pins the binding between the golden bytes and the signature, without introducing a + // second constant: the expected signature is derived from the golden payload string, so + // this fails if Sign ever hashes anything other than what BuildPayload returned. + const string key = "composition-canonical-payload-golden-test-hmac"; + var golden = Corpus["composition-populated"]; + using var hmac = new HMACSHA256(Encoding.UTF8.GetBytes(key)); + var expected = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(golden.Payload))); + + new CompositionCertificationRecordSigner(hmacKey: key).Sign(golden.Record).Should().Be(expected); + } + + private static string Sha256Hex(byte[] bytes) + { + using var sha = SHA256.Create(); + return Convert.ToHexString(sha.ComputeHash(bytes)); + } + + private static IReadOnlyDictionary LoadCorpus() + { + var path = Path.Combine(AppContext.BaseDirectory, "Tests", "Certification", GoldenFileName); + if (!File.Exists(path)) + throw new FileNotFoundException($"Golden composition payload corpus not found at '{path}'.", path); + + using var document = JsonDocument.Parse(File.ReadAllText(path)); + var options = new JsonSerializerOptions { PropertyNameCaseInsensitive = true }; + var corpus = new Dictionary(StringComparer.Ordinal); + foreach (var element in document.RootElement.GetProperty("cases").EnumerateArray()) + { + var name = element.GetProperty("name").GetString()!; + corpus[name] = new GoldenCase( + JsonSerializer.Deserialize(element.GetProperty("record").GetRawText(), options)!, + element.GetProperty("payload").GetString()!, + element.GetProperty("payloadSha256").GetString()!, + element.GetProperty("payloadByteLength").GetInt32()); + } + + return corpus; + } + + private sealed record GoldenCase( + CompositionCertificationRecord Record, + string Payload, + string PayloadSha256, + int PayloadByteLength); +} diff --git a/src/Ashlar.Tests.Infrastructure/Tests/Certification/composition-payloads.golden.json b/src/Ashlar.Tests.Infrastructure/Tests/Certification/composition-payloads.golden.json new file mode 100644 index 000000000..f5a586f05 --- /dev/null +++ b/src/Ashlar.Tests.Infrastructure/Tests/Certification/composition-payloads.golden.json @@ -0,0 +1,73 @@ +{ + "_readme": "Golden canonical signing payloads for COMPOSITION certification records (CompositionCertificationRecordSigner). These bytes are the message every composition admission signature is computed over, so they are pinned here and read by the in-repo golden test. To regenerate a case deliberately, copy the actual payload the failing test prints into this file. Regenerate ONLY when the canonical form is deliberately changed, and understand that changing it invalidates every composition signature already written. Doubles are deliberately restricted to exactly representable values: decimal formatting of arbitrary doubles is not identical across every target this tree builds for.", + "cases": [ + { + "name": "composition-populated", + "why": "Every field carrying a non-default value, plus the two the composition payload must structurally exclude (signature, gate). Structural mutant id lists are supplied out of order to pin the ordinal sort, and the reason string pins the escaping of quote, angle brackets, ampersand, apostrophe, plus and non-ASCII.", + "record": { + "status": "PASS", + "stage": "S0-S2", + "admitted": true, + "signed": true, + "timestamp": "2026-01-02T03:04:05.1234567+00:00", + "compositionId": "golden-composition", + "compositionEscapeRate": 0.5, + "totalStructuralMutants": 7, + "survivingStructuralMutants": 2, + "killedStructuralMutantIds": [ + "m-2", + "m-10", + "m-1" + ], + "survivingStructuralMutantIds": [ + "s-2", + "s-1" + ], + "signature": "SIGNATURE-MUST-NOT-APPEAR", + "reason": "golden fixture \u2014 quote \u0022 angle < & > apostrophe ' plus + caf\u00E9", + "gate": "GATE-MUST-NOT-APPEAR" + }, + "payloadSha256": "A46FC0A63E37C53DE6F7BF8424B79C43A0773682AB1EB7080C38E0F57C313307", + "payloadByteLength": 436, + "payload": "{\u0022status\u0022:\u0022PASS\u0022,\u0022stage\u0022:\u0022S0-S2\u0022,\u0022admitted\u0022:true,\u0022signed\u0022:true,\u0022timestamp\u0022:\u00222026-01-02T03:04:05.1234567Z\u0022,\u0022compositionId\u0022:\u0022golden-composition\u0022,\u0022compositionEscapeRate\u0022:0.5,\u0022totalStructuralMutants\u0022:7,\u0022survivingStructuralMutants\u0022:2,\u0022killedStructuralMutantIds\u0022:[\u0022m-1\u0022,\u0022m-10\u0022,\u0022m-2\u0022],\u0022survivingStructuralMutantIds\u0022:[\u0022s-1\u0022,\u0022s-2\u0022],\u0022reason\u0022:\u0022golden fixture \\u2014 quote \\u0022 angle \\u003C \\u0026 \\u003E apostrophe \\u0027 plus \\u002B caf\\u00E9\u0022}" + }, + { + "name": "composition-minimal", + "why": "Only the required members. Pins that the payload still carries all 12 names with null values and empty arrays - a small payload is legitimate, an empty one is not.", + "record": { + "status": "FAIL", + "stage": "S0", + "admitted": false, + "signed": false, + "timestamp": "2026-01-02T03:04:05+00:00", + "compositionId": "golden-minimal-composition" + }, + "payloadSha256": "96EDB0A99A329303A833EE66FE5911938D92411B91388E3B06E480D46770A7F1", + "payloadByteLength": 322, + "payload": "{\u0022status\u0022:\u0022FAIL\u0022,\u0022stage\u0022:\u0022S0\u0022,\u0022admitted\u0022:false,\u0022signed\u0022:false,\u0022timestamp\u0022:\u00222026-01-02T03:04:05.0000000Z\u0022,\u0022compositionId\u0022:\u0022golden-minimal-composition\u0022,\u0022compositionEscapeRate\u0022:null,\u0022totalStructuralMutants\u0022:null,\u0022survivingStructuralMutants\u0022:null,\u0022killedStructuralMutantIds\u0022:[],\u0022survivingStructuralMutantIds\u0022:[],\u0022reason\u0022:null}" + }, + { + "name": "composition-escaping", + "why": "Non-ASCII, a control character, a surrogate pair, a tab, and every character the default JavaScript encoder escapes - in compositionId and reason. Pins the encoder and its uppercase hex.", + "record": { + "status": "PASS", + "stage": "S0-S2", + "admitted": true, + "signed": true, + "timestamp": "2026-08-12T10:00:00+00:00", + "compositionId": "caf\u00E9-composition", + "compositionEscapeRate": 0.25, + "totalStructuralMutants": 1, + "survivingStructuralMutants": 0, + "killedStructuralMutantIds": [ + "m-1" + ], + "survivingStructuralMutantIds": [], + "reason": "caf\u00E9