Skip to content

The untyped-content gate reports a normal TRANSIENT degradation as a defect — the two seams disagree about the same event #3645

Closed

Description

@rbuergi

The two seams describe the same event and disagree about whether it is a failure

MeshNodeStreamCache warns at the instant of a read, and at that instant it cannot know the
type will register moments later. That is the normal state during portal boot — before the NodeType
compiles land — and it is precisely the race #2952 fixed, by re-typing every live reader when the
registration arrives.

The sibling seam one layer down already says so, in its own message:

MeshNodeTypeSource[…/Live]: content discriminator '$type': 'X' is not a registered type on the owning hub — content stays an untyped JsonElement **for now** … Two causes look identical here: (a) the NodeType's runtime compile has not registered 'X' **YET, which is TRANSIENT** … or (b) no declaration will ever claim this discriminator …

MeshNodeTypeSource passes no exception, so it never reaches _meshweaver-test-trace.log and
reds nothing. MeshNodeStreamCache (since #3625) passes MeshNodeContentDegradedException, so it
does. One event, two descriptions, opposite CI consequences.

Why this matters

check-untyped-content.sh reds a shard on any occurrence, so it currently reports a normal
transient
as a defect. Measured on #3628: two of LateContentTypeRegistrationTest's three cases
degrade and recover inside a single test — the recovery is the assertion — yet both produce a
gate-reddening record. A portal boot that reads an in-mesh NodeType's content before its compile
lands produces the same record for the same non-reason.

The gate is still worth having: a final degradation is a real, silent defect, and the gate's first
working run found one (Ops/Modules/{deployment}, fixed in #3628). The problem is the denominator —
a hit currently means "content was unreadable at a read", not "content is unreadable".

Why #3628 did not fix it

#3628's red was fixed minimally and correctly: the three tests capture their own degradation records
and assert them (see Doc/Architecture/ReadingCiSignals → "How a test declares that a
degradation is its own SUBJECT"
). Distinguishing transient from final is a different change — it
needs a window that nothing in a process naturally closes — so it is a rework rather than a repair,
and expanding #3628 into it would have been the wrong call.

Direction (a hypothesis, not a decision)

The platform already HAS the recovery event: #2952 wired IMeshContentTypeRegistry.Registrations to
re-type every live reader. So the emitter is not missing information, it is missing a place to
decide "this one never recovered". Candidates, none measured yet:

  • report at the recovery seam instead — emit the marker only when a degraded path is evicted /
    the reader completes without ever having been re-typed;
  • carry the verdict in the record (NodeTypeCompilationStatus at the time of the read), so the gate
    can key on Ok-and-still-untyped, which is the (b) the message already describes;
  • leave the emitter alone and give the GATE the second half — cross-reference each reported path
    against the NodeType's compilationStatus in the same run.

Acceptance

A gate hit means "content was unreadable and stayed unreadable", and a portal boot that reads a
runtime-compiled NodeType's content before its compile lands produces no gate hit — with a
falsification in both directions, as #3628 did.

Refs #3625, #2952, #3628.

Activity

  1. added 2 commits that reference this issue on Sep 7, 2026
  2. rbuergi commented on Sep 9, 2026

    @rbuergi
    ContributorAuthor

    Fixed by #3797, merged to main

    The gate's key moved from the event to the verdict.

    • check-untyped-content.sh now keys on MeshNodeContentUnresolvedException — emitted once per node type at MeshNodeStreamCache.Dispose, for entries that are still unresolvable when the mesh ends.
    • ContentDegradationRegistry.Unresolved(registry) computes that by re-asking the mesh-wide content-type registry per entry, by both routes TryRecoverForNodeType takes (two pure map lookups, no content retained — Record keeps the stored $type alongside the NodeType so the NAME route can be re-asked later).
    • The per-read MeshNodeContentDegradedException records stay in the logs and stay useful — once the verdict names a type, they are how you find which reads degraded on it — but they no longer red a shard on their own.

    So a portal boot that reads a runtime-compiled NodeType before its compile lands produces no gate hit, and a discriminator no declaration will ever claim produces exactly one, naming the node type and the discriminator. That is this issue's acceptance criterion.

    /health was wrong the same way and is fixed in the same change: ContentTypeHealthCheck fed Snapshot(), so a replica stayed Degraded after a boot race until some later read happened to clear the entry — a probe answering about the past.

    Falsified in both directions, as asked: reduce Unresolved to Snapshot() and the recovered case is reported again (ARecoveredDegradation_IsNotTheVerdict fails); remove the teardown report and AnUnrecoverableDegradation_IsTheVerdict_AndReachesTheSink fails on "the teardown must REPORT what never resolved". UntypedContentDegradationGate grew from 4 assertions to 6 and additionally pins that the report is computed from Unresolved( — a report built from Snapshot() would re-red every boot race invisibly, because the marker would still be constructed and the gate would still match.

    Doc/Architecture/ReadingCiSignals → TRANSIENT from FINAL is rewritten; it had recorded this as "needs a window nothing in a process naturally closes — a rework, deliberately left to its own issue". The answer turned out not to need a window at all: asking the registry again was enough.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions