Skip to content

analyze_server: an empty analysis window is "unavailable", not an all-clear (#3524) - #3553

Merged
erikdarlingdata merged 1 commit into
devfrom
fix/3524-false-all-clear
Sep 18, 2026
Merged

erikdarlingdata merged 1 commit into
devfrom
fix/3524-false-all-clear

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

What

analyze_server answered "No significant findings. All metrics are within normal ranges." for a server whose collection is dead. The analysis pipeline's data-span gate measures lifetime history, so a server monitored for a month with a credential that broke last Tuesday passes the gate, collects zero facts over the empty window, and the zero-facts pass returned a bare [] that the tool rendered as a true-negative all-clear — in prose, to an operator or their agent.

How

  • Both SKUs' analysis services (DarlingAnalysisService, Lite's AnalysisService) now set a distinguishable state, WindowEmptyMessage, beside the existing InsufficientDataMessage, at the zero-facts branch — naming the window, the total history that got it past the gate, and that nothing was measured. The branch also logs a warning (it was previously silent, including on scheduled passes).
  • Both SKUs' analyze_server tools check it before the findings-count branch and return the same unavailable envelope get_analysis_facts already uses for this exact state (DarlingMcpTools.cs:201-208 was the honest arm), telling the caller collection appears broken for this window and to check get_collection_health.
  • The all-clear is unchanged for the genuine case: facts collected, zero findings.
  • The unavailable envelope carries the same analysis_time/persisted/persistence_note hints as the all-clear, so an anchored empty-window run still discloses non-persistence (The analysis family cannot be anchored at a past window, and it is the one an incident starts with #2506 contract preserved — the existing AnalysisAsOfAnchorTests persistence assertions pass unmodified).

Evidence

New tests in both SKUs, run on the real 24h gate (not a zeroed one), covering both sides:

  • 25h of history whose newest row is 5h old (gate passes, default 4h window empty) → status: "unavailable", message points at get_collection_health, no "within normal ranges" anywhere, WindowEmptyMessage set, InsufficientDataMessage null.
  • Same server plus one benign wait inside the window → status: "empty" with the existing all-clear, WindowEmptyMessage null.

Test results (local):

  • Lite (Lite.Tests.exe, MTP): AnalysisWindowEmptyTests (2, new), AnalysisAsOfAnchorTests, AnalysisServiceTests, AnalysisBudgetTests, RemainingEmptyReadsToolTests — 27/27 pass; FactCollectorMiseryTests 23/23 pass.
  • Darling (Darling.Tests.exe, MTP, gated classes run live against a throwaway PG 18.4 + TimescaleDB 2.28.1 cluster): AnalyzeWindowEmptyLivePostgresTests (new), AnalysisAsOfAnchorLivePostgresTests, DarlingAnalysisPipelineTests, DarlingMcpToolsTests — 26/26 pass, 0 skipped; ViewerRecommendationsTests, TempDbCeilingStoreTests, DarlingWebEndpointsTests, AnalysisPlanAdvisoryGzReadTests, AnalysisPassCommandTimeoutTests — 94/94 pass.
  • Both builds: 0 Warning(s).

Out of scope (filed, not dropped)

The viewer Recommendations tabs (Lite RecommendationsTab, Darling Viewer) render the same "All clear — no current recommendations." from the same state and only branch on InsufficientDataMessage — that's #3551, deliberately outside this fix's file scope. The services' WindowEmptyMessage added here is what the viewers will consume.

Fixes #3524

🤖 Generated with Claude Code

https://claude.ai/code/session_01QMYFB4qv5tqGwMmdPsa4Zx

…-clear (#3524)

The analysis pipeline's data-span gate measures LIFETIME history, so a
server whose collection died (broken credential, unreachable target)
still passes it, collects zero facts over the empty window, and the MCP
tool rendered that as status "empty" with "All metrics are within
normal ranges" — an affirmative all-clear for a dead collector.

Both SKUs' analysis services now set WindowEmptyMessage (beside
InsufficientDataMessage) at the zero-facts branch, and both analyze
tools return the same "unavailable" envelope get_analysis_facts already
uses for this state, pointing the caller at get_collection_health. The
true-negative all-clear is unchanged for runs that collected facts and
found nothing. The unavailable envelope keeps the #2506 persistence
hints, so anchored empty-window runs still disclose non-persistence.

Tests, both SKUs, on the REAL 24h gate: 25h of history whose newest row
is five hours old -> unavailable; the same server with one benign wait
inside the window -> the existing all-clear. Darling's is gated
(DARLING_TEST_PG) and ran green against a live PG 18.4 + TimescaleDB
store; Lite's runs on the shared DuckDB fixture.

The viewer Recommendations tabs render the same false all-clear from
the same state and only branch on InsufficientDataMessage; that half is
#3551 (out of this fix's file scope).

Fixes #3524

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QMYFB4qv5tqGwMmdPsa4Zx
@erikdarlingdata
erikdarlingdata enabled auto-merge (squash) September 18, 2026 03:40
@claude

claude Bot commented Sep 18, 2026

Copy link
Copy Markdown

Reviewed. Change is symmetric between Lite (AnalysisService/McpAnalysisTools) and Darling (DarlingAnalysisService/DarlingMcpTools) — same field, same branch placement, same envelope shape. WindowEmptyMessage is correctly reset alongside InsufficientDataMessage at the top of AnalyzeAsync, the two are mutually exclusive (both branches return [] early), and dataSpanHours is in scope at the zero-facts branch in both files. New tests exercise the real 24h gate rather than a zeroed one, matching the PR's own claim.

One thing worth double-checking, not a blocker since the PR body already discloses it: Lite/Controls/RecommendationsTab.xaml.cs (and the Darling Viewer's RecommendationsViewModel) still branch only on InsufficientDataMessage and would render the same all-clear (FromItems([])) for a dead-collector window — i.e. the exact bug this PR fixes for the MCP tool still reproduces in the desktop UI. The PR calls this out explicitly as out-of-scope/filed as #3551, so flagging here only to confirm that's intentional and not an oversight, and that #3551 is tracked to land before this becomes a customer-visible gap in practice.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — verified Lite/Darling parity in the new WindowEmptyMessage field and its MCP-tool branch (identical placement, envelope, and reset logic in both AnalysisService/DarlingAnalysisService and McpAnalysisTools/DarlingMcpTools), checked variable scope (dataSpanHours) and mutual exclusivity with InsufficientDataMessage, and confirmed the referenced test helpers (ServerIdHelper, LiveStoreCleanup, CollectionIdGenerator, SharedDuckDbFixture) exist. No T-SQL touched. The one gap I found (viewer Recommendations tabs not yet updated) is explicitly disclosed in the PR body as filed under #3551, so it's not a hidden drift.

@erikdarlingdata
erikdarlingdata merged commit 1900967 into dev Sep 18, 2026
8 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/3524-false-all-clear branch September 18, 2026 03:50
erikdarlingdata added a commit that referenced this pull request Sep 18, 2026
…he false all-clear (#3551) (#3565)

The tabs only branched on InsufficientDataMessage, so an analysis window that
collected zero facts (dead collector, unreachable target) rendered "All clear -
no current recommendations." Both viewers now consume the WindowEmptyMessage
state #3553 added, one rung over from insufficient-data:

- Lite's Generate now reads AnalysisService.WindowEmptyMessage directly and
  renders a distinct notice pointing at the Collection Health tab.
- The Darling viewer never calls the engine, so the worker persists the
  window-empty determination into the V19 analysis_state marker as
  insufficient_data = false + the engine's message (a shape nothing else
  writes, so no schema change), and AnalysisStateMarker.WindowEmpty derives
  it back for the Recommendations tab on both the read and Generate now doors.

The genuine all-clear (facts measured, zero findings) is unchanged, and the
marker self-heals on the next facts-bearing pass - pinned through the real
viewer read against live Postgres.

Fixes #3551


Claude-Session: https://claude.ai/code/session_01QMYFB4qv5tqGwMmdPsa4Zx

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant