Repository navigation
Fix the alert notebook's collector-freshness check (#4378) - #4416
Merged
Merged
Conversation
- Freshness now counts only SUCCESS/SKIPPED collection_log rows, not ERROR runs, matching the other freshness reads in this codebase. - The freshness command uses the sibling reads' 20s CommandTimeout. - Elapsed time is measured with a Stopwatch instead of reporting 0. - The AG and Server Unreachable/Restored recovery pairs now resolve the notebook status to "Resolved at T" (AG Failover has no natural recovery edge and stays out). - Renamed a stray '(#4223 lane 0)' comment to '(#4223)'. - Test-only: AlertNotebookAuthoredTemplateTests' shared theories are now data-driven off AlertNotebookEndpoint.s_authoredTemplates so a future authored family needs no edit to this file.
erikdarlingdata
marked this pull request as ready for review
September 26, 2026 13:21
erikdarlingdata
deleted the
fix/4378-notebook-freshness-success-only
branch
September 26, 2026 13:21
This was referenced Sep 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #4378.
Why
The alert-notebook endpoint's freshness check decides between "No resolution recorded" and "Unknown (not collected since …)" for a metric whose alert history gave no direct answer. It had three problems: it counted ANY collector run including
ERRORrows as evidence the instrument was up, it ran without the sibling reads' command timeout, and its failure path always logged 0 ms elapsed regardless of how long the read actually took.What changes
status = 'SUCCESS'(see Freshness evidence below).CommandTimeout = McpCommandDeadlines.ReadSeconds(20s), the same budget the sibling MCP reads already use.Stopwatchand reportElapsedMillisecondson failure instead of a hardcoded0.AlertNotebookAuthoredTemplateTests.csto cite#4223only.AlertNotebookAuthoredTemplateTests' shared theories (validate / known-dispatch / compose-catalog / budget / binding / viz) were driven off a hand-written two-metric list (BothTemplates()). That list is nowAllAuthoredMetrics(), generated by flatteningAlertNotebookEndpoint.s_authoredTemplates, so a future authored family (Alert notebooks: authored templates for the remaining families, including custom rules and analysis findings #4223) is covered by these shared theories the moment its registration row lands, with no edit to this test file. The routing theory is now similarly data-driven, asserting each registered metric routes to its own row'sEntry.Id.Recovery edges
The AG family and the Server Unreachable/Restored pair fire under their own metric names rather than through a resolution-title alias, so
StatusFromHistory's arm 1 (which only consultedDarlingTriageEndpoint.ResolutionAliases) never reached them — a notebook opened against "AG Replica Disconnected" could not read "Resolved at T" even when "AG Replica Reconnected" arrived later.Added a small notebook-only pair list (
AlertNotebookEndpoint.NotebookRecoveryEdges) consulted by arm 1 alongsideResolutionAliases:This is deliberately NOT added to
ResolutionAliasesitself, because that list also buildsDarlingTriageEndpoint.SectionsByMetric, and every metric above already has its own entry there — adding them toResolutionAliaseswould define the same section-map key twice.AG Failoveris left out on purpose: it is an event (a role change), not a condition that clears, so it has no natural recovery edge.Freshness evidence
Only
SUCCESScounts as evidence the collector reached the target.SKIPPEDwas dropped from the freshness predicate: it can be written before any query reaches the target —Lite/Services/RemoteCollectorService.cs's user-cancelled-MFA catch setsstatus = "SKIPPED"at the point the sign-in is cancelled, never having contacted the server. Darling's own per-run classifier,EnumeratedCollectorDriver.ClassifyReturnedRun, never returnsSKIPPEDeither, so the status carries no server-reachability information on either code path.Test plan
AlertNotebookEndpointTests,AlertNotebookAuthoredTemplateTests,DocCommentHygieneTests— all pass in-process on this machine (32, 77, 29 total respectively, 0 failed).AlertNotebookEndpointTests.cs:CollectorFreshnessSqlcontains the status predicate and excludes'ERROR'; a source-text pin confirmsCommandTimeout = McpCommandDeadlines.ReadSecondsis set and that noReport(..., 0, ex)call remains for either read; three recovery-edge pins (AG reconnect, Server Restored, AG Failover absent from the pair list).AlertNotebookCollectorFreshnessLiveTests(3 facts), against my own throwaway container (timescale/timescaledb:2.30.1-pg18, removed after this run): only-ERROR-after-anchor reads Unknown; SUCCESS-after-anchor reads "No resolution recorded"; only-rows-before-anchor reads Unknown. All 3 GREEN on this branch.origin/dev(084cfb3) in a detached worktree: both the pure and the live test files fail to COMPILE against the old code (ResolveStatusAsync,CollectorFreshnessSql, andNotebookRecoveryEdgesdon't exist there —ResolveStatusAsyncwasprivateand is nowinternalfor the live test to call it directly).Darling.Tests.csprojandLite.Tests.csprojbuild 0 errors with-p:EnableWindowsTargeting=true.LivePostgresCollectionHygieneTests.EveryClassUsingTheSharedStore_IsSerializedOrDocumentsWhyNotfails on this Mac with aPresentationFrameworkload error — pre-existing platform gap unrelated to this change (it reflects over all loaded assemblies, which includes a WPF-referencing assembly this environment cannot load); left for CI, which runs on Windows.Darling.Tests/Lite.Teststargetnet10.0-windowsand cannot fully run here; CI decides the rest of the suite.CHANGELOG entry
SECTION: Fixed
ENTRY:
REF:
[Fix the alert notebook's collector-freshness check (#4378) #4416]: Fix the alert notebook's collector-freshness check (#4378) #4416