Skip to content

A server dark past collection_log's retention reads Offline everywhere, not 'Awaiting first collection' (#3967) - #3975

Merged
erikdarlingdata merged 1 commit into
devfrom
feature/3967-dark-past-retention
Sep 23, 2026
Merged

erikdarlingdata merged 1 commit into
devfrom
feature/3967-dark-past-retention

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

Closes #3967.

Why

#3966 (#3935) taught the fleet card that "nothing found" is not "never collected". Every other surface that reads a server's newest collection still made the same mistake, at a longer horizon.

list_servers, the WPF sidebar dot and the WPF Overview card read that collection with no window. But collection_log keeps only CollectionLogRetentionDays (60) days. A server dark for longer than that has no row left, and ClassifyFreshness(null) calls it never collected. So a decommissioned server left enabled read Offline for two months, then turned amber: "Awaiting first collection — registered, but the service has not reached it yet".

Lite has the same case. Its Overview card reads hot, then the archive view, and the archive files are deleted after ArchiveRetentionMonths (3). A server dark past that read "Never", with an amber "No collection has ever landed for this server".

What changes

One rule, in PerformanceMonitor.Common. New overload:

ServerHealthClassifier.ClassifyFreshness(lastCollectionUtc, registeredAtUtc, searchedFromUtc, nowUtc)
  • A newest collection bands on the ladder, exactly as before.
  • With none, a registration before searchedFromUtc (the earliest instant the read could see) is Offline.
  • A registration at or after it keeps the ladder's never-collected reading. That half is exact: registration happens when collection can first begin, so any collection would be inside what the read can see.
  • No registration also keeps the ladder's reading.

The callers are the surfaces that previously called the ladder with a null last collection:

Surface Registration Horizon it passes
Fleet card (#3966's ClassifyWindowedFreshness, now a delegate) servers.created_date on the probe row its 48 h window start
list_servers (ServerListSql + created_date, FreshnessStatus) servers.created_date DarlingRetentionHorizons.CollectionLogHorizon(now) = now − 60 d
WPF sidebar dot (DarlingServer.ApplyFreshness) DarlingServer.RegisteredAt, from created_date on BOTH ServersSql and ManagedServersSql same
WPF Overview card (ServerSummaryItem.ApplyFreshness) stamped by the loader from the registry row, like IsPostgres same
Lite Overview card (GetServerSummaryAsync → ApplyCollectionFreshness) ServerConnection.CreatedDate → RegisteredAtUtc (not persisted) RetentionService.OldestRetainedInstant(now)

Lite's horizon is a month start, not the cleanup's cutoff. Archive files are named for the month (or day) they were written, and CleanupOldArchives deletes a file once that date is before the cutoff, so a month's file goes whole. A row lands in a file of its own month or later, so it is certain to survive only from the first month start at or after the cutoff. A test runs the real cleanup against that month and the month before it, so the two cannot drift.

Lite needs the registration passed in, so it is now required. GetServerSummaryAsync(id, name, registeredAtUtc) takes it as a required parameter, and all three callers pass one:

  • the Overview;
  • list_servers;
  • get_server_summary, through a new ServerResolver.RegisteredAtUtc.

So the band never depends on which caller asked.

Text that assumed the old meaning of an empty read:

  • WPF Overview "Last collection" cell: Never → None retained when the card is Offline with nothing to show.
  • Lite Overview "Last Collect" cell: Never → None retained (stopped).
  • WPF Manage Servers "Last Collected" column: Never → None retained. The dialog now reads the registry once for the registration instants.
  • WPF server tab toolbar: no data collected yet → no collection retained.
  • Lite list_servers line: No data collected → none retained.

No token or payload shape changes. The list_servers status token for such a server moves from AwaitingFirstCollection to Offline, and nothing else changes. No new fields, no description change. The MCP payload's last_collection stays null for the case.

Not changed: no migration, and no new reads anywhere except the Manage Servers dialog's one registry read. DarlingServer's new constructor parameter is defaulted, so the test fakes compile unchanged.

Where two surfaces can still differ, deliberately. A server that connected more than 48 hours ago but inside the retention, and has never written a single collection_log row, reads Offline on the fleet card and "Awaiting first collection" on the unbounded surfaces. The fleet read cannot see far enough back to prove it never collected; the others can. It is rare, because every collector run writes a row whatever its outcome. Every server that ever collected now reads the same on every surface: the live test asserts one answer per server across list_servers, the dot, the card and the fleet card.

Test plan

  • DARLING01 (read-only):
    • ServerListSql before and after, warm: 0.79 ms / 0.81 ms execution, 29 buffers each, same plan. created_date rides the registry row it already reads.
    • The session's first, cold run spent 172.8 ms planning (9,694 buffers). That is the unbounded MAX planning every chunk, a cost this PR does not touch.
    • ManagedServersSql after: 0.48 ms.
    • No registry row on DARLING01 predates the 60-day horizon (the store is 18 days old), so nothing there changes reading today.
  • New DarkPastRetentionReadsOfflineTests (Darling, 11 pure):
    • the shared rule over every registration and reading, and its boundary;
    • the horizon;
    • the fleet card delegating to the rule;
    • list_servers through RenderServerList;
    • created_date on all three registry reads;
    • the dot, the card and the Manage Servers cell;
    • fresh, stale and offline readings unchanged on every surface;
    • the loader stamping the registration before banding.
  • New DarkPastRetentionReadsOfflineLivePostgresTests (rig 55501, TimescaleDB). Four registered servers: history aged out, registered 10 min ago, dark 5 days inside retention, and collecting. They are read through list_servers, both viewer registry reads (config seeded, so ManagedServersSql really runs), the sidebar freshness read, the Overview summary read, and GetFleetOverviewAsync. Each surface gets the expected word, and all four surfaces agree for every server.
  • New DarkPastRetentionReadsOfflineToolTests (Lite, 7):
    • the month-start horizon;
    • the horizon against the real CleanupOldArchives;
    • the shipped summary read for aged-out, new and unregistered servers;
    • a retained collection still shown;
    • the registration from servers.json, not persisted;
    • ServerResolver.RegisteredAtUtc;
    • list_servers text for both cases.
  • Updated deliberately: three Lite test call sites and Lite's EveryCardIsStampedWithItsBandWhereItIsBuilt source pin, which now pins the stamp WITH the registration.
  • Red-watch, test projects rebuilt each time. Five mutations, each caught:
    • the shared rule disabled: 9 Darling failures and 2 Lite;
    • the dot dropping its registration: 2;
    • list_servers dropping it: 2;
    • ManagedServersSql's reader dropping the column: 1 (live);
    • Lite's summary dropping it: 2.
  • Full Darling.Tests with DARLING_TEST_PG on the rig: 12,712 total, 1 failed, 24 skipped. The failure is CollectionHealthAggregateTests.InstallHole_FirstPolicyRunAfterEnsure_StrandsNoHistory_AgainstDevPostgres: 55P03 could not refresh continuous aggregate "collection_health_hourly" due to a concurrent refresh, a race with the rig's background refresh job on a class this PR does not touch. Re-run alone, the class passes 13/13. Every skip is environment-gated: managed runtime or upgrade fixtures, a live SQL Server, symlink privilege, or CI sizing.
  • Full Lite.Tests: 5,139 total, 0 failed, 0 skipped.
  • Both builds: 0 Warning(s).

🤖 Generated with Claude Code

https://claude.ai/code/session_018Z4T51J4E6L8fjriPHpsif

…e, not "Awaiting first collection" (#3967)

#3966 fixed the fleet card's 48-hour read. The surfaces that read a server's
newest collection with no window still called a null "never collected":
list_servers, the WPF sidebar dot and Overview card, and Lite's Overview card.
Retention bounds what they can see (collection_log 60 days, Lite's archive 3
months), so a server dark for longer turned amber.

The rule now lives once in PerformanceMonitor.Common:
ServerHealthClassifier.ClassifyFreshness(lastCollection, registeredAt,
searchedFrom, now). A null on a server registered before what the read could
see is Offline; otherwise the ladder decides. The fleet card delegates with its
48 h window start. The unbounded Darling surfaces pass
DarlingRetentionHorizons.CollectionLogHorizon(now). Lite passes
RetentionService.OldestRetainedInstant(now), the first month start the
archive still holds whole.

Registration comes from servers.created_date: on ServerListSql, and on both
ServersSql and ManagedServersSql via DarlingServer.RegisteredAt, which the
Overview loader stamps onto the card. In Lite it comes from
ServerConnection.CreatedDate, a required parameter on GetServerSummaryAsync.

The "Never" / "No data collected" texts now say "None retained" for that case.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Z4T51J4E6L8fjriPHpsif
@erikdarlingdata
erikdarlingdata merged commit 3197633 into dev Sep 23, 2026
15 of 16 checks passed
@erikdarlingdata
erikdarlingdata deleted the feature/3967-dark-past-retention branch September 23, 2026 02:24
erikdarlingdata added a commit that referenced this pull request Sep 23, 2026
…h the archive cleanup keeps, not at the cutoff day (#3984)

The cleanup deletes an archive file whole once the month it is named for
falls before the cutoff, so the days between the cutoff and the next month
start are gone with it. DailySummaryRetentionHorizon returned the cutoff day,
which judged those days as retained with no data instead of purged, and
published a retention_horizon up to a month earlier than Lite's history. It
now returns RetentionService.OldestRetainedInstant, the instant #3975 gave
the Overview freshness band.


Claude-Session: https://claude.ai/code/session_01GdmA4ND1wLSqA91ax1m4xv

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
erikdarlingdata added a commit that referenced this pull request Sep 23, 2026
…3920, #3927, #3931, #3932, #3940, #3942, #3946, #3947, #3950, #3952, #3955, #3956, #3957, #3964, #3965, #3966, #3968, #3972, #3975, #3979, #3980, #3981, #3983, #3984, #3985) (#3989)

The wave's fix PRs deliberately carried no CHANGELOG edits (parallel-agent hot-spot protocol); each agent reported its entry and this commit lands them together, byte-verified against origin/dev.


Claude-Session: https://claude.ai/code/session_01GdmA4ND1wLSqA91ax1m4xv

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant