Repository navigation
Revert cadence-aware fleet staleness (#3268): it deletes the amber band fleet-wide - #3270
Merged
Merged
Conversation
Reverts PR #3268 and its CHANGELOG entry (#3269). Field review against the pgmon store showed the change is a net regression: EffectiveStaleThreshold takes the max over a server's collectors, so any enabled slow-cadence collector (these PG targets carry 9 hourly + 4 daily) pushes the per-server stale cutoff to tens of hours, while ClassifyFreshness checks the flat 30-minute Offline band first. That leaves the amber Stale band unreachable on every real target: a server reads Fresh until 30 minutes fully dark, then jumps to Offline, so a multi-minute collection gap now shows green. The original #3236 signature was a true positive (a real coverage collapse during the nightly restart), not the false positive the fix assumed, because last_collection is MAX over all collectors and the ~70s collectors keep it current on a healthy server. Reopening #3236 to re-diagnose with fleet measurement in the loop. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dqza1QNL4YCFS1MpzezERn
|
Reviewed. This is a clean, symmetric revert of #3268 across Verification performed:
No correctness, parity, or security issues found in the revert itself. No T-SQL changed (pure C#/CHANGELOG diff), so the T-SQL style conventions don't apply here. |
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reverts #3268 (and its CHANGELOG entry from #3269). Reopens #3236.
Why
A field review against the pgmon store found #3268 is a net regression, verified from source before this revert:
The amber band becomes unreachable fleet-wide.
ServerHealthClassifier.EffectiveStaleThresholdtakes the max over a server's enabled collectors, and a current collector contributes ~its own cadence (StaleThresholdFor= 2x cadence). Real PostgreSQL targets carry 9 hourly + 4 daily collectors, so the daily ones alone hold the per-server stale cutoff at >=24h (measured 40h15m live). ButClassifyFreshnesschecks the flat 30-minute Offline band first, then the stale threshold. With the stale cutoff at 40h > 30min, the amberStalewindow[stale, offline)is empty: a server reads Fresh until 30 minutes fully dark, then jumps to Offline. A multi-minute collection gap — the exact thing #3236 was filed about — now shows green. That fails toward the quiet label, which is the wrong direction for a staleness signal. The change's own doc comment admits the no-amber outcome but scopes it to "only slow-cadence collectors enabled";maxmakes it "any slow-cadence collector present," which is every real target.The original diagnosis was wrong.
FleetLastCollectionSqlisMAX(collection_time)over all collectors, so on a healthy server the ~70-second collectors keep it current and the flat 2-minute threshold never trips spuriously. The 14-target amber event in #3236 was a real ~4-minute coverage collapse during the nightly service restart — a true positive the band reported correctly — not the false positive the fix assumed.What's reverted
All 20 code/test files from #3268 (the
EffectiveStaleThresholdpath across the fleet overview,list_servers, the viewer cards/dots, and Lite's Overview card) plus the 2-line CHANGELOG entry. CHANGELOG census verified CRLF-only, entry removed. Pre-#3268 flat-threshold behavior is restored.#3236 is reopened for re-diagnosis with fleet measurement in the loop; any real fix must pin that the amber band stays reachable (
EffectiveStaleThreshold(...) < OfflineThreshold) over a representative collector mix.🤖 Generated with Claude Code
https://claude.ai/code/session_01Dqza1QNL4YCFS1MpzezERn