Repository navigation
Lite has no persisted alerted-state memory, so every edge-triggered database state still repeats forever #2203
Description
Parity gap left open by #2166 / PR #2182, filed so it is tracked rather than living only in a changelog caveat. Surfaced again while fixing #2189.
What is missing
#2182 made the four "somebody chose this" database states (OFFLINE, RESTORING, RECOVERING, STANDBY) edge-triggered: they alert on the transition and stay quiet until the state changes. The memory that makes that work is persisted per database in config.database_state_expected.last_alerted_state / last_alerted_at (V60), and the engine compares against it:
var alreadyAnnounced =
DatabaseStateTokens.RepeatsAreNoise(db.StateDesc)
&& string.Equals(db.LastAlertedState, db.StateDesc, StringComparison.OrdinalIgnoreCase);Lite's config_database_state_expected has no such columns (Lite/Database/Schema.cs), LiteAlertStateStore.SaveDatabaseStateAlertedAsync / ClearDatabaseStateAlertedAsync are no-ops, and Lite's deviation read returns no LastAlertedState. So in Lite alreadyAnnounced is always false, every deviation reads as new, and behavior is exactly pre-#2166: a database parked OFFLINE for a month produces an alert every cooldown, forever. That is the original reported complaint, still live in Lite.
This is deliberate and documented (AlertEngine.cs: "A host that does not persist the memory (Lite today) reports empty here"), not an oversight. It just has no issue.
Why it is worth doing rather than leaving
While fixing #2189 the review turned up a scenario where the missing memory turns a mild annoyance into the same flood shape #2189 was about:
- A database is OFFLINE when first observed, so the seed learns OFFLINE as its baseline (nobody chose it, but it is a steady state, so it is legitimately learned).
- Somebody brings it up for an hour of maintenance and re-parks it.
- In Darling,
RepeatsAreNoise(OFFLINE)is true and the persisted memory edge-triggers it: one alert. - In Lite, the same sequence alerts every cooldown for as long as it stays parked.
#2189 deliberately does NOT heal an auto-baselined OFFLINE row (healing it would be a worse bug - see that PR), so this path stays reachable in Lite until the memory lands.
Shape of the fix
Mirror V60 in DuckDB and wire the three pieces Darling already has, which are the same three every new store-backed field needs:
- Two nullable columns on
config_database_state_expected(last_alerted_state,last_alerted_at). LiteAlertStateStore.SaveDatabaseStateAlertedAsync/ClearDatabaseStateAlertedAsyncbecome real writes (UPDATE, never upsert -PgAlertStateStoredocuments why an INSERT here would baseline a SUSPECT database as expected-SUSPECT and silence it permanently, andAlertEngineTests.DatabaseState_AlertedStamp_IsAnUpdate_NeverAnInsertpins it).LocalDataService.GetDatabaseStateDeviationsAsyncselects the column intoDatabaseStateInfo.LastAlertedState, and gains the store-derived recovery clear thatDarlingAlertReadAdapter.ClearRecoveredDatabaseStateAlertsSqlowns - that one is load-bearing, since the engine's own clear runs off an in-memory set that empties on restart.
The six review findings on #2182 are all behavioral and all worth re-testing on the Lite side, especially: a muted fire must not record the state as announced (or the mute becomes permanent), recovery must clear the memory (or a second parking never alerts), and the cooldown must be keyed per state (or an OFFLINE-to-SUSPECT flip inside one window is swallowed).
Activity
erikdarlingdata commentedon Aug 11, 2026
Taking this one — checked for an existing
fix/2203-*branch or PR first and there is none, so claiming it here before starting rather than after. (I raced you on #2189 by not doing that; not repeating it.)It's the natural continuation of #2182 on my side: I built the Darling half, wrote the "Lite parity follows in its own change" caveat you're quoting, and left the no-ops. Thank you for filing it — you're right that a caveat living only in a changelog is not tracking.
Your OFFLINE-maintenance escalation is the part that changes my read of the priority. I had it filed in my head as "Lite behaves as it did before #2166," which sounds like a non-regression. But the sequence you describe — a database legitimately baselined OFFLINE, brought up for an hour, re-parked — produces the #2189 flood shape in Lite while Darling emits one alert, and that is a materially worse thing than "unchanged."
Scope I'm planning, so you can tell me if I've mis-shaped it:
config_database_state_expected(last_alerted_state,last_alerted_at) plus the schema-version bump.LiteAlertStateStore.SaveDatabaseStateAlertedAsync/ClearDatabaseStateAlertedAsyncimplemented against DuckDB instead of no-op'd.LastAlertedStatesoalreadyAnnouncedcan be true there.activeset empties on restart, so a restart between an alert and the recovery leaves the memory sticky forever and swallows the next episode. Darling clears from what the store holds; Lite needs the same or it inherits the bug I already fixed once.DatabaseStateExpectedStoreTests(in-process DuckDB, no live gate) covering the edge trigger, the repeat-episode-after-recovery case, and the mute case — a muted fire must not record as announced, or a temporary mute becomes permanent.Will post here if that shape is wrong or if I hit anything that changes it.
Lite gets the persisted alerted-state memory (#2203, the #2166 Lite h…
Merge pull request #2204 from erikdarlingdata/lite-alerted-state-memo…
erikdarlingdata commentedon Aug 11, 2026
Landed on dev in #2204 (merge
2e38447b), rebased onto #2202's #2189 fix and green on all five checks with a genuine review (5m07s, real token spend, no skip signature, no blocking findings).What shipped: Lite's
config_database_state_expectedgainslast_alerted_state/last_alerted_at(schema v53, both nullable — NULL means never announced, which is what a first observation, a fresh store and a recovered database all look like, and all three must stay free to alert);SaveDatabaseStateAlertedAsyncandClearDatabaseStateAlertedAsyncactually write instead of being documented no-ops; and the deviation read returns the value so the sharedAlertEngine'salreadyAnnouncedcomparison — which lives once, inPerformanceMonitor.Alerting, and is not reimplemented per SKU — finally sees something other than false in Lite.The clear is store-derived rather than taken only from the engine's in-memory active set, for the reason the Darling half (#2166) had to learn: that set empties on restart, so a restart landing between an alert and the recovery left the memory stuck forever and silently swallowed the next parking of that database. The engine's immediate falling-edge call is kept as well, so a recovery inside one process doesn't wait for the next sweep.
Unchanged on purpose: the integrity states (SUSPECT, RECOVERY_PENDING, EMERGENCY) still re-fire on the cooldown. Nobody parks a database in SUSPECT, and there the repetition is the signal.
Closing — the PR targeted dev, which doesn't auto-close.
Stop asserting a best-effort maintenance cycle always runs (#2266 ite…
Stop asserting a best-effort maintenance cycle always runs (#2266 ite…
Report a skipped database-state maintenance cycle (#2266 follow-on) (#…
Database-state writes take the WRITE lock (erikdarlingdata#2208)
Fix two aged-database migration bugs behind erikdarlingdata#2748's Li…