Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,27 @@ which defect — is on the second line. A diagnosis that arrives one line too la

Each is a distinct first body line, so each cause arrives as its own incident naming its own cause.

### …and its own EVENT ID, because a recurrence folds by SITE (#1549, second half)

A distinct template gives each class its own *fingerprint* — but the incident watcher also folds
every fingerprint of one log **site** into a single issue (`ComputeSiteFold`), and a site is part 1
plus part 2 above: category + event id + exception type. The template is not in it. With every class on
one category, event id `0` and no exception, the classes were still one site, so after the template
split `NO NODE EXISTS` (2026-09-24) and `OWNING HUB REACHED NO VERDICT` (2026-09-25) still arrived on
#1549 as "a different fingerprint of the same log site". That issue was about silent forwards, fixed by
#1990, and these were neither.

So `LogReleaseRefusal` logs each class under an event id of its own:
`NodeTypeRecompileExtensions.ReleaseFailureEventId`, a **permanent, explicit code per class** in the 7400 range (`ReleaseFailureEventIds`), named
`NodeTypeRelease<Class>` (the console line shows it in the bracket: `fail:
MeshWeaver.Graph.NodeTypeRecompileExtensions[7405]` is `NodeMissing`). The codes are never the enum ordinal: a code is the identity of a
deployed incident site, so inserting a class mid-enum must not hand it a historical class's id and
renumber the rest. A new class takes the next unused code (append only, never reuse).
`TheEventIdOfEveryClass_IsPinned` pins the whole mapping, so a missing, moved or reused code fails, and
`EveryFailureClass_LogsUnderAnEventIdOfItsOwn` checks that the ids are distinct. An unlisted value takes
`Unclassified`'s id, the same way it takes its template. The negative
control is to remove the ids, which leaves every class on one shared id.

### Classify at the arm, never from the text

🚨 **The class is decided where the exception still is.** Recovering it downstream by matching
Expand Down
85 changes: 69 additions & 16 deletions src/MeshWeaver.Graph/NodeTypeRecompileExtensions.cs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
using System.Collections.Immutable;
using System.Reactive.Linq;
using MeshWeaver.Graph.Configuration;
using MeshWeaver.Mesh;
Expand Down Expand Up @@ -42,6 +43,57 @@ public static class NodeTypeRecompileExtensions
/// plus per-type seeds; on elapse every affected type simply compiles, as it always did.</summary>
private static readonly TimeSpan PrebuiltSeedBudget = TimeSpan.FromSeconds(60);

/// <summary>
/// 🚨 <b>Every failure class logs under an <see cref="EventId"/> of its OWN</b> (#1549, second
/// half). The class moved into the template so a reader and a fingerprint can tell the causes
/// apart — but the incident watcher FOLDS every fingerprint of one log SITE into a single issue,
/// and a site is keyed on category + event id + exception type, never on the template. With all
/// templates sharing one category, event id 0 and no exception, they were still one site:
/// <c>NO NODE EXISTS</c> (2026-09-24) and <c>OWNING HUB REACHED NO VERDICT</c> (2026-09-25) both
/// arrived on #1549 as "a different fingerprint of the same log site", 459 occurrences of causes
/// that issue was never about. One id per class makes one site per class.
///
/// <para>🚨 <b>The ids are PERMANENT and EXPLICIT, never the enum ordinal</b> (review on #6326): an
/// id is the identity of a deployed incident site, so a class inserted mid-enum must not take a
/// historical class's id and renumber the rest. Each class is listed in
/// <see cref="ReleaseFailureEventIds"/> with its code; a new class gets the next unused code and
/// <c>ReleaseFailureClassIsInTheTemplateTest</c> pins the whole mapping, so a missing entry or a
/// reused or moved code fails there. An unlisted or undeclared value logs under the
/// <see cref="NodeTypeReleaseFailure.Unclassified"/> id, exactly as it takes that template.</para>
/// </summary>
/// <param name="failure">The classified failure.</param>
/// <returns>The event id that failure is logged under.</returns>
internal static EventId ReleaseFailureEventId(NodeTypeReleaseFailure failure) =>
ReleaseFailureEventIds.TryGetValue(failure, out var id)
? new EventId(id, $"NodeTypeRelease{failure}")
: new EventId(ReleaseFailureEventIds[NodeTypeReleaseFailure.Unclassified],
$"NodeTypeRelease{NodeTypeReleaseFailure.Unclassified}");

/// <summary>
/// The permanent class → event-id codes (range 7400–7499; the disposal verdicts use 73xx). Append
/// only: never renumber, never reuse a retired code.
/// </summary>
internal static readonly ImmutableDictionary<NodeTypeReleaseFailure, int> ReleaseFailureEventIds =
new Dictionary<NodeTypeReleaseFailure, int>
{
[NodeTypeReleaseFailure.Unclassified] = 7400,
[NodeTypeReleaseFailure.NoNodeTypePath] = 7401,
[NodeTypeReleaseFailure.CompileDenied] = 7402,
[NodeTypeReleaseFailure.PermissionCheckFailed] = 7403,
[NodeTypeReleaseFailure.PermissionCheckNoVerdict] = 7404,
[NodeTypeReleaseFailure.NodeMissing] = 7405,
[NodeTypeReleaseFailure.OwnerUnreachable] = 7406,
[NodeTypeReleaseFailure.OwnerRecycling] = 7407,
[NodeTypeReleaseFailure.WriteDenied] = 7408,
[NodeTypeReleaseFailure.WriteRejected] = 7409,
[NodeTypeReleaseFailure.BaseStateNeverArrived] = 7410,
[NodeTypeReleaseFailure.HostTearingDown] = 7411,
[NodeTypeReleaseFailure.StorageUnavailable] = 7412,
[NodeTypeReleaseFailure.TransientHubFailure] = 7413,
[NodeTypeReleaseFailure.NoAnswerWithinBound] = 7414,
[NodeTypeReleaseFailure.HostLeaving] = 7415,
}.ToImmutableDictionary();

/// <summary>
/// 🚨 The release-refusal log line, ONE TEMPLATE PER FAILURE CLASS — issue #1549.
///
Expand Down Expand Up @@ -92,80 +144,81 @@ internal static void LogReleaseRefusal(ILogger? logger, NodeTypeReleaseRefusal r
return;
var path = refusal.NodeTypePath;
var message = refusal.Reason;
var eventId = ReleaseFailureEventId(refusal.Failure);
switch (refusal.Failure)
{
case NodeTypeReleaseFailure.NoNodeTypePath:
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — NO NODETYPE PATH was supplied: {Message}",
path, message);
break;
case NodeTypeReleaseFailure.CompileDenied:
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — COMPILE DENIED to the caller: {Message}",
path, message);
break;
case NodeTypeReleaseFailure.PermissionCheckFailed:
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — the COMPILE PERMISSION CHECK COULD NOT RUN, so nothing was decided: {Message}",
path, message);
break;
case NodeTypeReleaseFailure.PermissionCheckNoVerdict:
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — the COMPILE PERMISSION CHECK ENDED WITHOUT A VERDICT, neither granting nor denying: {Message}",
path, message);
break;
case NodeTypeReleaseFailure.NodeMissing:
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — NO NODE EXISTS at that path: {Message}",
path, message);
break;
case NodeTypeReleaseFailure.OwnerUnreachable:
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — the OWNING HUB REACHED NO VERDICT on the trigger write: {Message}",
path, message);
break;
case NodeTypeReleaseFailure.OwnerRecycling:
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — the OWNING ACTIVATION WAS RECYCLING, so the write never applied: {Message}",
path, message);
break;
case NodeTypeReleaseFailure.WriteDenied:
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — the OWNER DENIED the trigger write: {Message}",
path, message);
break;
case NodeTypeReleaseFailure.WriteRejected:
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — the OWNER REJECTED the merged value: {Message}",
path, message);
break;
case NodeTypeReleaseFailure.BaseStateNeverArrived:
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — NO INITIAL STATE for the node ever arrived, so the write had nothing to diff against: {Message}",
path, message);
break;
case NodeTypeReleaseFailure.HostTearingDown:
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — the HOST WAS TEARING DOWN under the in-flight write: {Message}",
path, message);
break;
case NodeTypeReleaseFailure.StorageUnavailable:
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — the DATA STORE COULD NOT BE REACHED: {Message}",
path, message);
break;
case NodeTypeReleaseFailure.TransientHubFailure:
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — a TRANSIENT HUB OR ROUTING MISS ended the write: {Message}",
path, message);
break;
case NodeTypeReleaseFailure.NoAnswerWithinBound:
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — the RELEASE LEG NEVER ANSWERED within its ordered bound: {Message}",
path, message);
break;
case NodeTypeReleaseFailure.HostLeaving:
logger.LogWarning(
logger.LogWarning(eventId,
"[Recompile] Release request for {Path} not issued — THIS HOST IS LEAVING, so nothing it sends can reach the owner: {Message}",
path, message);
break;
Expand All @@ -174,7 +227,7 @@ internal static void LogReleaseRefusal(ILogger? logger, NodeTypeReleaseRefusal r
// 🚨 Named, not absorbed. An arrival here says a real shape has no rule yet — which
// is a finding about the classifier, and must not be filed as any of the classes
// above. It is the one template that is expected to shrink to zero traffic.
logger.LogError(
logger.LogError(eventId,
"[Recompile] Release request for {Path} failed — UNCLASSIFIED trigger-write fault, no rule claimed its shape: {Message}",
path, message);
break;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,69 @@ public void EveryFailureClass_GetsATemplateOfItsOwn()
.Select(g => string.Join("+", g.Select(t => t.Failure)))));
}

/// <summary>
/// 🚨 Distinct templates were not enough (#1549, second half): the incident watcher folds every
/// fingerprint of one log SITE into one issue, and a site is category + event id + exception type.
/// Every class shared event id 0, so after the templates split, a missing node and an
/// owner that reached no verdict still landed on #1549 as recurrences. Every class must therefore
/// log under an event id of its own, and none under 0.
///
/// <para>Negative control, watched red: with the ids removed from the calls every class reports
/// event id 0 and this fails with one distinct id for every class.</para>
/// </summary>
[Fact]
public void EveryFailureClass_LogsUnderAnEventIdOfItsOwn()
{
var classes = Enum.GetValues<NodeTypeReleaseFailure>();
var ids = classes.Select(failure => (Failure: failure, Id: Log(failure, "reason").EventId)).ToArray();

ids.Should().OnlyContain(x => x.Id.Id != 0,
"event id 0 is the shared site every class used to fold into");
ids.Select(x => x.Id.Id).Distinct().Should().HaveCount(classes.Length,
"one event id per class, so one incident site per class. Shared: "
+ string.Join(", ", ids.GroupBy(x => x.Id.Id).Where(g => g.Count() > 1)
.Select(g => string.Join("+", g.Select(x => x.Failure)))));
ids.Should().OnlyContain(x => x.Id.Name == $"NodeTypeRelease{x.Failure}",
"the id's name is the class, so a log line names its cause even where the template is not shown");
}

/// <summary>
/// 🚨 The ids are deployed incident SITES, so they are pinned exactly — never derived from the enum
/// ordinal (review on #6326). A class inserted mid-enum must not take a historical class's id, and
/// a new class must be given its own code: this test fails on a missing entry, a moved code, or a
/// reused one. Append a row when adding a class; never edit an existing row.
/// </summary>
[Fact]
public void TheEventIdOfEveryClass_IsPinned()
{
var pinned = new Dictionary<NodeTypeReleaseFailure, int>
{
[NodeTypeReleaseFailure.Unclassified] = 7400,
[NodeTypeReleaseFailure.NoNodeTypePath] = 7401,
[NodeTypeReleaseFailure.CompileDenied] = 7402,
[NodeTypeReleaseFailure.PermissionCheckFailed] = 7403,
[NodeTypeReleaseFailure.PermissionCheckNoVerdict] = 7404,
[NodeTypeReleaseFailure.NodeMissing] = 7405,
[NodeTypeReleaseFailure.OwnerUnreachable] = 7406,
[NodeTypeReleaseFailure.OwnerRecycling] = 7407,
[NodeTypeReleaseFailure.WriteDenied] = 7408,
[NodeTypeReleaseFailure.WriteRejected] = 7409,
[NodeTypeReleaseFailure.BaseStateNeverArrived] = 7410,
[NodeTypeReleaseFailure.HostTearingDown] = 7411,
[NodeTypeReleaseFailure.StorageUnavailable] = 7412,
[NodeTypeReleaseFailure.TransientHubFailure] = 7413,
[NodeTypeReleaseFailure.NoAnswerWithinBound] = 7414,
[NodeTypeReleaseFailure.HostLeaving] = 7415,
};

Enum.GetValues<NodeTypeReleaseFailure>().Should().OnlyContain(f => pinned.ContainsKey(f),
"every class needs a pinned code here — a class without one would log under the "
+ "Unclassified site and fold into it");
foreach (var (failure, id) in pinned)
Log(failure, "reason").EventId.Id.Should().Be(id,
$"{failure}'s event id is a deployed incident site and must never move");
}

/// <summary>
/// The fallback is NAMED and reached only by <see cref="NodeTypeReleaseFailure.Unclassified"/>.
/// A fallback that some other class can also reach is a class that silently means "and
Expand Down Expand Up @@ -313,7 +376,7 @@ private static Captured Log(NodeTypeReleaseFailure failure, string reason)
+ "that logs two doubles every count built on it").Which;
}

private sealed record Captured(LogLevel Level, string Template, string? Path, string? Message);
private sealed record Captured(LogLevel Level, string Template, string? Path, string? Message, EventId EventId);

/// <summary>
/// Captures the TEMPLATE, from <c>{OriginalFormat}</c> — not the formatted string. The formatted
Expand All @@ -336,7 +399,8 @@ public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Except
logLevel,
Value(values, "{OriginalFormat}") ?? string.Empty,
Value(values, "Path"),
Value(values, "Message")));
Value(values, "Message"),
eventId));
}

private static string? Value(IReadOnlyList<KeyValuePair<string, object?>> values, string key)
Expand Down