Repository navigation
issue-3653 A8 option B: a live guard that no anomaly detector logs an error (silent-family guard) - #4179
Merged
Conversation
…dow (#3653 A8 option B) Both PgAnomalyDetector (SQL Server store) and PgTargetAnomalyDetector (PG target) swallow their own per-family exceptions into a logged Error line, so a SQL error or a missing parameter bind silently kills a family with no fact and no failed test. This adds Darling/Darling.Tests/AnomalyDetectorErrorGuardLiveTests.cs: a capturing ILogger runs every family over minimally seeded tables (one row per table each *WindowSql*/*TileWindowSql* const reads, across a 4h and a 24h window) and fails if any Error-or-above line was captured. Part of #3653
…ane G1-2) Seed 12 same-hour/day-of-week rows per family table (12 weeks back) instead of 1, so every detector's baseline.SampleCount clears BaselineMath.CollapseThreshold and each family reaches its *WindowSql read instead of returning early on SampleCount == 0. Capture the SQL Npgsql actually executed via CommandCapture.CaptureAsync and assert every public const string *WindowSql field on PgAnomalyDetector / PgTargetAnomalyDetector (reflected, so it covers every partial-class piece automatically) appears verbatim among the captured commands, naming any that never ran.
…gap (lane G1-3) - RunBothWindowsAsync now returns Task<bool> (CommandCapture.CaptureAsync needs Func<Task<T>>, not Func<Task>) — the merge with dev (#4171) broke the build. - AssertEveryWindowSqlRan excludes PgTargetAnomalyDetector's SessionWindowSql, CpuWindowSql, WaitRateWindowSql by name: #4171's tiled-window recipe gave those arms tile twins the detector actually reads now; the plain consts stay only for PgTargetAnomalyTests' pinning. PgAnomalyDetector's SQL Server consts of the same names are NOT excluded (still read directly there). - SeedSqlServerFamilyTablesAsync: added a spiked current-window wait_stats row so the wait-profile detector's fallback bar actually fires and WaitContribWindowSql runs (a flat series across every seeded row never exceeded the bar).
…ard's 4 window consts WIP: DatabaseCounterWindowSql, WaitContribWindowSql and SampledWaitContribWindowSql now run; IoLatencyWindowSql still needs a fix (baseline gate: PgIoReadLatency SampleCount==0).
…row (still gapped) DatabaseCounterWindowSql, WaitContribWindowSql and SampledWaitContribWindowSql now run live. IoLatencyWindowSql still 'never ran' -- the baseline arm's per-15-minute date_bin bucket collapses same-bucket rows via MAX before LAG runs, so a same-bucket predecessor/successor pair does not give LAG anything to diff against. Needs a predecessor in the PRECEDING 15-minute bucket, not the same one. Left for a follow-up; documented in the PR body.
…sor in the preceding 15-minute bucket
The current-window pg_io_stats pair sat 30 seconds apart, inside the
SAME date_bin('15 minutes') bucket, so the sampled CTE's MAX()
aggregation collapsed them to one row per bucket before LAG ever ran
-- raw_reads stayed NULL every time and IoLatencyWindowSql never
executed, exactly the silent-family failure this guard exists to
catch. Move the pair 15 minutes apart (one bucket earlier, then the
bucket itself) with an increasing reads/read_time_ms counter so LAG
resolves a real predecessor.
Part of #3653
erikdarlingdata
force-pushed
the
feat/3653-b-detector-error-guard
branch
from
September 24, 2026 20:23
c42df0c to
3216c5d
Compare
erikdarlingdata
marked this pull request as ready for review
September 24, 2026 20:23
…mments Two real CI failures after the dev rebase, neither a flake: 1. AssertEveryWindowSqlRan reported CpuBurnWindowSql and WalVolumeWindowSql as never-ran. Same situation as the three consts already excluded (SessionWindowSql/CpuWindowSql/WaitRateWindowSql, from lane L3a): L3c (#4178) declared these two only so the census keeps seeing the same SQL text PgTargetFactCollector uses to build the PG_CPU_BURN_CORES/ PG_WAL_VOLUME_SHIFT facts -- the detector itself only ever calls their tile twins, CpuBurnTileWindowSql/WalVolumeTileWindowSql. Added both to the existing supersededByTile exclusion set. 2. DocCommentHygieneTests.NoMemberCarriesTwoStackedSummaryBlocks: two <summary> blocks stacked on SeedSqlServerFamilyTablesAsync (247/252) and SeedPgTargetFamilyTablesAsync (349/353). Checked each pair first, per that test's own warning -- both describe the SAME member, the first an earlier/shorter draft ("one row... background history row") superseded by a later, more complete one written when a lane raised the seeding from 1 row to 12 (explaining the CollapseThreshold floor). Deleted the stale first block in each case; nothing lost, the second already carries everything the first said and more. Local: real dotnet build clean; DocCommentHygieneTests class run clean (0 failed). The guard tests themselves skip locally (no DARLING_TEST_PG) -- Windows CI's live PG service is the arbiter for the fixture-gap fix.
Closed
39 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #3653.
Final status: guard fully green, all three lane steps done (G1-5)
This lane (G1-5) closed the last gap G1-4 left open:
IoLatencyWindowSqlnever ran because its baseline arm bucketspg_io_statsat a 15-minutedate_bingrain withMAX()beforeLAG, and the seeded current-window pair sat 30 seconds apart — inside the SAME bucket — so the sampled CTE's ownMAX()aggregation collapsed them to one row beforeLAGever ran.raw_readsstayedNULLon every attempt.Step 1: seed fix
Moved the current-window
pg_io_statspair to 15 minutes apart (at.AddMinutes(-15)andat, matching the const's owndate_bin('15 minutes', ...)grain) with an increasingreads/read_time_mscounter, soLAGresolves a real predecessor from the preceding bucket rather than the same one. Ran the class live — bothNoAnomalyDetectorLogsAnErrorOverASeededWindow_SqlServerStoreand..._PgTargetgreen, every*WindowSqlconst (includingIoLatencyWindowSql) confirmed run viaAssertEveryWindowSqlRan:Step 2: red-first proof, both directions
(a) Corrupted SQL — the guard names the SQL error. Changed
IoLatencyWindowSql'sMAX(reads)toMAX(reads_typo). Result:(b) Missing history — the guard names the const that never ran. Commented out the two
IoLatencyWindowSqlcurrent-window seed inserts (the fix from step 1). Result:Both mutations reverted (verified
git diffclean against the committed fix before rebuilding), rebuilt with--no-incremental, and re-ran green (2/2) before continuing.Step 3: full suite, once
Full
Darling.Tests.dllrun withDARLING_TEST_PGunset:215 failed = the pre-existing macOS-WPF-execution-environment baseline this lane's brief names as expected (
214 = the base— I measured 215 here; the delta is environmental noise unrelated to this change, none of the failures are inAnomalyDetectorErrorGuardLiveTests,PgTargetAnomalyTests,PgTargetIoTests, orPgTargetBetweenWavesV2Tests). No new failures introduced by this lane's change.What I did NOT verify
DARLING_TEST_PGset (the live PG class only, per the brief's targeted-iteration instruction); Lite.Tests was not built/run on this Mac (WPF, per house rig instructions — left to CI).Live run verified
timescale/timescaledb:2.28.1-pg18container on port 55891,DARLING_TEST_PGset. Container removed after the run (docker rm -f g15-pg).CHANGELOG entry
(none — tests only)