Skip to content

0 warnings in tile behaviour tests + wait-profile reads its tile SQL once - #4185

Merged
erikdarlingdata merged 3 commits into
devfrom
fix/3653-a8-tile-hygiene
Sep 25, 2026
Merged

erikdarlingdata merged 3 commits into
devfrom
fix/3653-a8-tile-hygiene

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Part of #3653 (A8 option B residue).

Why

The tile work in #4172 and #4177 left two defects on dev:

  1. Six xUnit2031 build warnings in the two tile-behaviour test files. Each passes x.Where(predicate) to Assert.Single instead of using its predicate overload.
  2. A rider from the client-site loop seat. It said the wait-profile detector in PgAnomalyDetector (DetectWaitAnomalies) reads its per-hour tile SQL twice in each analysis pass. That detector emits ANOMALY_WAIT_PROFILE. Lite's twin reads the SQL once.

What changes

1. The six xUnit2031 warnings

Each Assert.Single(facts.Where(f => f.Key == "X")) is now Assert.Single(facts, f => f.Key == "X"), which returns the same element. The sites are SqlServerStoreTileBehaviourTests.cs lines 103, 197, 322 and 418, and SqlServerStoreTileBehaviourBatchQueryMemoryTests.cs lines 98 and 197. Darling.Tests now builds with 0 Warning(s).

2. The rider: confirmed and fixed

DetectWaitAnomalies ran WaitRateTileWindowSql, the tiled wait-rate read, through two separate commands:

  • rateCmd built the WindowTile list.
  • totalCmd ran the same SQL again, only to sum total_wait_ms (ordinal 3).

The source comment said the sum came "from the same rows the tiles came from". In fact the second command fetched them again. Lite's twin (Lite/Analysis/AnomalyDetector.cs, DetectWaitAnomalies, lines 508-514) has always summed that column in the one loop that reads the tiles.

The fix moves the sum into the tile-read loop, in the same shape as Lite: the same rateReader variable and the same loop body. No other detector in PgAnomalyDetector.cs has this pattern. The CPU, I/O, batch, session, query-duration and memory detectors each read their tiled window once. The fired facts do not change: both commands ran the same SQL with the same parameters. The fix halves the number of round trips for this read.

DarlingAnomalyBaselineTests.WaitRateWindow_ReadsThePeakAndMeanPair_AndTheRobustArmIsTheSharedPairGate_LiteVerbatim (from #4172) already pinned this difference as it was. Its comment called it "a real structural difference the coordinator has not yet ruled symmetric". This fix is that ruling. The pin now requires the single-read shape (rateReader on both products) and fails if the old shape's totalReader comes back.

Evidence

A local rig with pg_stat_statements preloaded counted the calls to WaitRateTileWindowSql in one full DetectAnomaliesAsync pass. The counters were reset before the pass, and the count matched a column alias that only this SQL uses.

Calls to WaitRateTileWindowSql per pass
Before this fix 2
After this fix 1

Lite's side was checked by reading the code, not by a live count. Its DetectWaitAnomalies has one command, one reader and one loop, which builds tiles and adds up windowTotalWaitMs. Lite was already right, and Darling now matches it.

Which test gates CI

The live count test moved into its own file and its own scratch PostgreSQL database: WaitRateTileReadCountLiveTests.Waits_DetectAnomaliesAsync_ReadsWaitRateTileWindowSqlOnce_NotTwice, carrying the #1776 own-store comment instead of [Collection("live-postgres")]. A CI run on the original shape (36082244687, job "Darling PG tests (2)") failed it against the SHARED store with 55000: pg_stat_statements must be loaded via "shared_preload_libraries". It then failed that collection's residue check (#1873). CREATE EXTENSION IF NOT EXISTS pg_stat_statements succeeds whether or not the library is preloaded, so the test's own skip, gated only on that CREATE, never fired. The extension got created and was left behind on a store where CI's darling-pg job preloads only timescaledb.

The test now reads current_setting('shared_preload_libraries') BEFORE creating anything and skips unless pg_stat_statements is listed, so a rig without the library never runs CREATE EXTENSION at all. CI still skips it the same way the file's other live tests skip when DARLING_TEST_PG is unset. The coordinator's ruling to leave CI's preload list unchanged stands. The test now runs on its own scratch database, dropped whole on dispose. So the shared collection's residue check never sees anything it did, and it no longer needs to serialize against the other live classes.

pg_stat_statements' counters are cluster-wide, keyed by dbid, not private to one database. Both the reset (pg_stat_statements_reset(0, dbid, 0)) and the count read now filter on the scratch database's own oid. A class running concurrently against a different database cannot inflate or zero this count out from under it that way.

The source pin in DarlingAnomalyBaselineTests is unchanged by this fix. It runs on every CI run and fails if the old two-reader shape comes back, and that is what gates CI. The live test is the independent, round-trip-counted proof, on any rig that preloads pg_stat_statements.

Test plan

  • dotnet build Darling/Darling.Tests/Darling.Tests.csproj -c Debug: 0 warnings, 0 errors, after every commit.
  • Rig WITHOUT pg_stat_statements preloaded (port 55982, only timescaledb, matching CI's darling-pg job exactly):
    • WaitRateTileReadCountLiveTests: 1 of 1, SKIPPED, not failed: pg_stat_statements is not in shared_preload_libraries ('timescaledb').
    • SqlServerStoreTileBehaviourTests: 5 of 5 passed, no LiveStoreResidueException from the collection fixture.
  • Same rig, restarted with pg_stat_statements added to shared_preload_libraries:
    • WaitRateTileReadCountLiveTests: 1 of 1 passed.
    • Reverted the one-read fix in PgAnomalyDetector.cs locally, not committed, then restored it right after. Against the reverted double-read code the test went red: Assert.Equal() Failure: Expected: 1, Actual: 2. Rebuilt after restoring the fix (0 warnings) and it passed again.
    • WaitRateTileReadCountLiveTests + SqlServerStoreTileBehaviourTests + DarlingAnomalyBaselineTests, run together: 25 of 25 passed.
  • The full suite was not run locally. CI runs it.

The rig ran PostgreSQL 18.6 with TimescaleDB (and, for the second half above, pg_stat_statements), extracted from the repo's pg-runtime.zip, at C:\GitHub\worktrees\rig-law2. It was stopped and the rig directory deleted at the end of the lane.

No CHANGELOG entry. The warning fix is test-only, and the detector fix changes no output.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TszxYhJJbTEh4LrZ56NYo3

erikdarlingdata and others added 3 commits September 24, 2026 21:27
Assert.Single(facts.Where(f => f.Key == "X")) triggers xUnit2031 (prefer
the predicate overload). Rewritten to Assert.Single(facts, f => f.Key ==
"X") at all 6 sites across SqlServerStoreTileBehaviourTests.cs and
SqlServerStoreTileBehaviourBatchQueryMemoryTests.cs. Same returned
element, Darling.Tests now builds with 0 Warning(s).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TszxYhJJbTEh4LrZ56NYo3
…SQL once, not twice

Rider ruling: confirmed. DetectWaitAnomalies (ANOMALY_WAIT_PROFILE) ran
WaitRateTileWindowSql twice per analysis pass - once via ReadTilesAsync to
build the tile list, once more, word for word, in a second NpgsqlCommand
just to sum total_wait_ms. Lite's twin (AnomalyDetector.DetectWaitAnomalies)
has always summed that column inside the same loop that reads the tiles.
Live-measured on a rig with pg_stat_statements preloaded: 2 calls to the
statement per pass before this fix, 1 after (SqlServerStoreTileBehaviourTests'
new Waits_DetectAnomaliesAsync_ReadsWaitRateTileWindowSqlOnce_NotTwice proves
it, and was red against the reverted double-read code before this commit).

Merges the two reads into the loop that already walks every row, matching
Lite's shape exactly (same rateReader variable, same loop). Fired-fact
values are unchanged - both reads always returned identical rows.

DarlingAnomalyBaselineTests' WaitRateWindow_ReadsThePeakAndMeanPair_...
pin (from #3653 B / #4169) explicitly deferred this exact asymmetry
"pending a coordinator ruling" - updated it to require the new symmetric
shape and forbid the old second-reader tell (totalReader) from coming back.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TszxYhJJbTEh4LrZ56NYo3
…ts own scratch store

CI run 36082244687 (job "Darling PG tests (2)") failed
Waits_DetectAnomaliesAsync_ReadsWaitRateTileWindowSqlOnce_NotTwice with
55000 pg_stat_statements must be loaded via shared_preload_libraries,
then failed the live-postgres collection's residue check (#1873): the
test ran CREATE EXTENSION IF NOT EXISTS pg_stat_statements against the
SHARED store, which succeeds whether or not the library is preloaded -
only the later pg_stat_statements_reset() / view read throws 55000, so
the test's own skip (gated on the CREATE alone) never fired, and the
extension it did manage to create was left behind on a store CI's
darling-pg job preloads only timescaledb for.

Moves the test out of SqlServerStoreTileBehaviourTests (shared
live-postgres collection) into its own file, WaitRateTileReadCountLiveTests,
on a scratch database (#1776 own-store): TryEnableAsync,
ConvertToHypertablesAsync, EnsureCollectionLogHypertableAsync,
EnsureContinuousAggregatesAsync, the same order the other own-store
live tests use. The skip now reads
current_setting('shared_preload_libraries') BEFORE creating anything,
so a rig without the library never runs CREATE EXTENSION at all - and
whatever this test does create lives only in the scratch database,
dropped whole on dispose, so the shared collection's residue check
never sees it either way.

pg_stat_statements' counters are cluster-wide, keyed by dbid, not
private to one database, so both the reset and the read now filter on
this scratch database's oid (pg_stat_statements_reset(0, dbid, 0)) -
without that, a class running concurrently against a different
database could inflate or reset this count out from under it.

DarlingAnomalyBaselineTests' source pin (the CI gate for this fix)
is unchanged.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TszxYhJJbTEh4LrZ56NYo3
@erikdarlingdata erikdarlingdata changed the title DO NOT MERGE (A8 hygiene): 0 warnings in tile behaviour tests + wait-profile reads its tile SQL once 0 warnings in tile behaviour tests + wait-profile reads its tile SQL once Sep 25, 2026
@erikdarlingdata
erikdarlingdata marked this pull request as ready for review September 25, 2026 02:03
@erikdarlingdata
erikdarlingdata merged commit ca81eb1 into dev Sep 25, 2026
18 of 19 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/3653-a8-tile-hygiene branch September 25, 2026 02:03
erikdarlingdata added a commit that referenced this pull request Sep 25, 2026
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TszxYhJJbTEh4LrZ56NYo3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant