Skip to content

Trace-flag display reads anchor on the collector's newest successful run (#3999) - #4030

Merged
erikdarlingdata merged 1 commit into
devfrom
fix/3999-trace-flags-collection-log-anchor
Sep 23, 2026
Merged

erikdarlingdata merged 1 commit into
devfrom
fix/3999-trace-flags-collection-log-anchor

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

Closes #3999.

Why

get_trace_flags (MCP, DarlingCurrentConfigReader.TraceFlagsSql), the web viewer's Trace Flags grid
(ViewerDataService.Config.TraceFlagsSql), and Lite's equivalent (LocalDataService.Config.cs
GetLatestTraceFlagsAsync) all anchored on capture_time = (SELECT MAX(capture_time) FROM v_trace_flags WHERE server_id = $1). This is correct for the ordinary case (one flag turns off while others stay on:
the off flag's stale row has an OLDER capture_time than the new MAX, so it is excluded correctly). It
fails on the edge case #3929's background also named for the analysis fact: a capture that finds EVERY
flag off writes ZERO rows (DBCC TRACESTATUS(-1) only ever lists flags that are ON), so MAX(capture_time)
cannot see that capture at all and silently falls back to an older capture that still had a flag on,
reporting a flag as currently enabled days or weeks after it (and every other flag) was turned off.

Erik's ruling (posted on the issue): anchor the display reads on the trace_flags collector's newest
SUCCESSFUL run in collection_log, not on the newest row. If that run is newer than the newest
trace_flags row, no flag is on, making the all-off case exact.

What changes

Each of the three reads gained a second AND: the newest trace_flags row's own timestamp must be
>= the newest SUCCESS this collector logged in v_collection_log. When a later successful run
found nothing on, that comparison is false, the whole WHERE matches no row, and the read correctly
reports zero flags instead of falling back to a stale one.

AND   (SELECT MAX(capture_time) FROM v_trace_flags WHERE server_id = $1) >= COALESCE(
          (SELECT MAX(collection_time) FROM v_collection_log
           WHERE server_id = $1 AND collector_name = trace_flags AND status = SUCCESS),
          TIMESTAMP 1900-01-01)

(quoted string literals in the actual code; shown unquoted above only to keep this body's markdown
simple to render)

The COALESCE floor matters only when collection_log's retention has aged past this collector's oldest
trace_flags row, or nothing has run yet: with no successful run to corroborate against, it falls back
to the pre-fix reading rather than wrongly suppressing a row it cannot compare.

  • Darling/PerformanceMonitor.Darling.Service/Mcp/DarlingCurrentConfigReader.cs - TraceFlagsSql
    (get_trace_flags).
  • Darling/PerformanceMonitor.Darling.Viewer/ViewerDataService.Config.cs - TraceFlagsSql (the web
    viewer's Trace Flags grid).
  • Lite/Services/LocalDataService.Config.cs - GetLatestTraceFlagsAsync (Lite parity, see below).
  • The analysis fact (PgFactCollector.Config.cs's TraceFlagsSql, and Lite's
    DuckDbFactCollector.Config.cs twin) is not touched. It already uses a different, independently
    correct technique from On-load config collectors also recapture daily, not just on connect (#3929) (#3930) #4001 (an on-load-aware lookback WINDOW rather than an exact match), which the
    issue's own ruling only asked to revisit "if it lets the window fallback go", a conditional
    simplification of already-correct code, not a defect, and out of scope for a hard-deadline lane.
    Flagging as a possible small follow-up rather than doing it speculatively under time pressure.

Lite parity

Checked Lite/Services/LocalDataService.Config.cs GetLatestTraceFlagsAsync against both Darling
reads: byte-identical defect (DuckDB SQL, same capture_time = MAX(capture_time) shape, same
v_trace_flags / v_collection_log view names). Fixed with the identical additional AND clause, DuckDB
syntax being PostgreSQL-compatible here (COALESCE, scalar subquery, TIMESTAMP literal all work
unchanged).

Test plan

  • Darling.Tests builds with 0 Warning(s) (includes the new live test).
  • Lite.Tests not rebuilt as part of this PR's local run, see below. The Lite change is a
    3-line SQL addition mirroring the already-verified Darling SQL byte-for-byte in shape, so I judged
    the risk low against the remaining time budget. Coordinator/CI should confirm Lite.Tests builds
    clean; flagging rather than asserting it silently.
  • New live regression test, Darling.Tests/ViewerConfigurationTests.cs:
    TraceFlags_AllOffSinceNewerSuccessfulRun_ReadsNoFlags_AgainstDevPostgres. Seeds one trace flag
    ON two days ago, then a collection_log SUCCESS for trace_flags one day after that (simulating the
    daily recapture that found nothing on and wrote no row), and asserts GetLatestTraceFlagsAsync
    returns EMPTY. This is exactly the bug shape: before the fix this would have returned the
    two-day-stale ON flag.
  • Existing live test TraceFlags_LatestCaptureWins_OrderedByFlag_AgainstDevPostgres re-read: it
    plants no collection_log row at all, so the COALESCE floor keeps it passing unchanged; the
    ordinary multi-flag case is untouched.
  • Existing SQL-substring pins (ViewerConfigurationTests.TraceFlagsSql_SelectsSnapshotColumns...,
    DarlingMcpConfigToolsTests.TraceFlagsSql_LatestSnapshot) use Assert.Contains, not an exact
    match, against the original capture_time = (SELECT MAX...) clause I preserved verbatim ahead of
    the new AND; re-read to confirm they still hold, not just assumed.
  • Re-read the other two live fixtures that plant trace_flags rows with no matching collection_log
    row (DarlingMcpConfigToolsTests.ConfigTools_ReadPlantedSnapshots_AgainstDevPostgres,
    McpLatestSnapshotStampTests config section); both stay correct under the COALESCE floor for the
    same reason.
  • Targeted class run (build machine has no live Postgres, so live facts self-skip via
    Assert.SkipWhen inside the run rather than being excluded by filter): Darling.Tests filtered to
    ViewerConfigurationTests, DarlingMcpConfigToolsTests, McpLatestSnapshotStampTests: 35 total, 0
    failed.
  • The new live test has NOT been executed against a real Postgres. This lane had no time left in
    its 13:10Z window to stand up the PG rig after On-load collectors are now staleness-exempt forever in CollectorHealthClassifier, even though they recapture daily #4000. Written and reasoned through carefully (see
    "Why" above) but UNVERIFIED live. This is the single most important thing for the coordinator to
    check before treating this PR as done; please run it against the CI darling-pg job or the local
    rig before arming auto-merge.
  • No dedicated Lite/DuckDB live regression test added for the all-off case, for the same reason.
    The source fix mirrors Darling's exactly; a DuckDB equivalent of the new live test (using
    SharedDuckDbFixture, following McpLatestSnapshotStampTests.cs's pattern) is the natural follow-up
    if the coordinator wants matching live coverage on both SKUs.

What the coordinator should double-check

  • Run the new live test for real (TraceFlags_AllOffSinceNewerSuccessfulRun_ReadsNoFlags_AgainstDevPostgres);
    it is reasoned through but not executed, per the Test plan above.
  • Lite.Tests build was not re-run locally after the LocalDataService.Config.cs edit; low risk (an
    additive SQL clause plus one doc comment, same shape as the verified Darling changes) but not proven.
  • Whether the analysis fact's window-based TRACE_FLAGS read (On-load config collectors also recapture daily, not just on connect (#3929) (#3930) #4001) is worth simplifying onto this same
    collection_log anchor, per the issue's own "if it lets the window fallback go"; left untouched here.
  • No store migration; this is a SQL-clause change against existing tables/views, no schema change.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GdmA4ND1wLSqA91ax1m4xv

…run, not the newest row (#3999)

get_trace_flags (MCP), the web viewer's Trace Flags grid, and Lite's equivalent all anchored on
capture_time = MAX(capture_time) over trace_flags alone. A capture that finds every flag off writes
ZERO rows (DBCC TRACESTATUS(-1) only lists flags that are ON), so that MAX cannot see the capture at
all and silently falls back to an older capture that still had a flag on - reporting a flag enabled
days or weeks after it, and every other flag, was turned off.

Each read now also requires the newest trace_flags row to be at least as new as the newest SUCCESSFUL
collection_log run for the trace_flags collector. When a later successful run found nothing on, that
comparison fails and the read reports no flags - exact, rather than a stale fallback. A COALESCE floor
('1900-01-01') preserves the old reading when collection_log has no matching row at all (its retention
aged past the oldest trace_flags row, or nothing has run yet), so the fix cannot suppress a real
result it has no evidence against.

Fixed in DarlingCurrentConfigReader.TraceFlagsSql (get_trace_flags), ViewerDataService.Config.TraceFlagsSql
(the web viewer), and Lite's LocalDataService.Config.cs GetLatestTraceFlagsAsync (parity).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdmA4ND1wLSqA91ax1m4xv
@erikdarlingdata
erikdarlingdata merged commit 42445e6 into dev Sep 23, 2026
15 of 16 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/3999-trace-flags-collection-log-anchor branch September 23, 2026 12:50
erikdarlingdata added a commit that referenced this pull request Sep 23, 2026
… one after verifying it (#4033)

On 2026-09-23 three lane PRs opened ready (#4020, #4022, #4030) were merged
from the UI before the coordinator's verification finished, and #4030 put a
regression on dev (every enabled trace flag hidden; fixed in #4032). A draft
can't be merged from the UI, so a lane now opens a draft, lists anything it
didn't run as an unchecked box, and the coordinator readies it after the
checklist read (and a review round for security PRs).


Claude-Session: https://claude.ai/code/session_01GdmA4ND1wLSqA91ax1m4xv

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
erikdarlingdata added a commit that referenced this pull request Sep 23, 2026
… its timestamp, so an ordinary run no longer hides every enabled flag (#4032)

#4030 compared MAX(trace_flags.capture_time) against the newest SUCCESS's
collection_time. Darling stamps collection_log when a run ENDS, after its
capture rows, so that comparison was false after every ordinary run and hid
every enabled flag in get_trace_flags and the viewer grid. Lite stamps at run
START, so the same SQL meant something different there.

All three reads (Darling MCP, Darling viewer, Lite) now keep the newest capture
unless the newest SUCCESS wrote 0 rows. Every capture writes the full list of
flags that are on, so 0 means all off. No SUCCESS row, or a NULL count, keeps
the pre-fix reading.

- Darling live test seeded in the service's real write order (capture, then an
  end-stamped log): on, all off, a failed run, on again, through BOTH readers.
  Red-watched: #4030's SQL fails step 1 with Expected [1117], Actual [].
- New Lite twin, TraceFlagsLatestRunTests, seeded in Lite's order (log
  stamped at start).


Claude-Session: https://claude.ai/code/session_01GdmA4ND1wLSqA91ax1m4xv

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
erikdarlingdata added a commit that referenced this pull request Sep 23, 2026
…ntries in their sections (#4080)

Adds 42 entries and 42 link refs (#3992, #3995, #3996, #3998, #4001, #4002, #4003, #4007, #4010, #4011, #4013, #4015, #4020, #4022, #4025, #4029, #4030, #4031, #4036, #4038, #4039, #4040, #4044, #4047, #4048, #4049, #4050, #4051, #4055, #4061, #4063, #4064, #4065, #4066, #4067, #4068, #4069, #4070, #4071, #4073, #4074, #4078). Each PR's entry was buffered, and this lands every entry whose PR was merged on origin/dev when it ran.

#3989 left 26 entries under bare 'Changed' and 'Fixed' lines above '### Added'. They move into '### Changed' and '### Fixed', below the new entries, and one blank line stays under [Unreleased].


Claude-Session: https://claude.ai/code/session_01Ua31ugERL5DmhFVRtf6keQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant