Repository navigation
An alert-pass read outside AlertPassCommandTimeoutTests' six-file allowlist is untimed, and the pin cannot see it #2932
Description
Activity
Claude posting for Erik Darling
Both halves are on
devas of14919d58(#2934).The site.
DarlingWorker.ReadLatestCpuAsynctakesDarlingAlertReadAdapter.AlertPassCommandTimeoutSecondsin the object-initializer form. One production line.The derivation, confirmed rather than inherited by proximity, which is what this issue asked for. The ceiling transfers exactly — 30 s
s_alertSweepInterval, a property of the pass, and this site's only caller isEvaluateAlertsAsyncunder the plainstoppingTokenwith no enclosingCancelAfter. The floor is now measured for this query rather than borrowed from the 1,744.9 ms forced-plan read: read-onlypsqlon the use2 store, running the SQL extracted from source and kept parameterized throughPREPARE/EXECUTE, gives 11.5–17.2 ms over six round trips on the heaviest server, 13.9 msEXPLAIN (ANALYZE)execution, and avg 12.5 ms / worst 50.6 ms across all 61server_ids cold. So 10 s clears the measured fleet-wide worst by ~200×.That measurement also disproved something this issue and I both expected. The issue anticipated "a single-row read on a covered ordering… fast", stated as an expectation. It is fast, but not for that reason: the plan is
Limitover a top-NSortover aChunkAppendof 29 compressed chunks, scanning 47,752 rows and 2,228 shared buffers to return one row, withChunks excluded during startup: 0because the query carries nosample_timepredicate. Brute force, not an ordering. Left untouched here and worth its own issue — asample_time > now() - interval '…'predicate would make it nearly free, and this runs per server per 30 s tick.The scope. Option one, not the filename list. Mixed-regime files are named by their entry point and the swept members are the transitive closure of calls out of
EvaluateAlertsAsync, computed per run — so the pin's claim is literally "runs insideEvaluateAlertsAsync". The six filenames survive only as a whole-file sweep of the six dedicated files, where whole-file is strictly broader than any closure over them. The measured reason the cheaper option was not taken: the same scanner over all ofDarlingWorker.csfinds 8 command sites, 6 untimed and every one owned by another budget, so a file-scoped guard would fail on correct code six times over.Red-first on this site, as required. Reverting the fix fails
EveryAlertPassCommandReachedFromAnEntryPoint_SetsAnExplicitDeadlinenamingDarlingWorker.cs ReadLatestCpuAsync. And because a widened scope that still could not see the file would pass while guarding nothing, the walk itself is proven: cutting the call — leaving the site present and correctly timed — fails the closure's positive control, a mutation under which every occurrence count in the file is invariant.Review during the PR found two further holes in that walk, both fixed and both proven both directions: expression-bodied members were invisible (three were already live on this call graph, method-group referenced), and nested declarations were swept twice.
Not closing #2874 — other regime groups remain there. One follow-up is filed as #2942: #2938's two-span call-site pattern needs applying to this pin, which could not be done here because
ConstructionSpanFromwas not ondevyet.
An alert-pass store read sits outside
AlertPassCommandTimeoutTests' six-file allowlist, so it is untimed and the pin is structurally unable to say soFound by #2874's group B and verified independently on current
dev. Two halves, filed together on purpose: fixing the site alone leaves the next one just as invisible.The untimed site
Darling/PerformanceMonitor.Darling.Service/DarlingWorker.cs:3886,ReadLatestCpuAsync:No
CommandTimeout, so it inherits Npgsql's undocumented 30 s default — the defect class #2874 exists to sweep.It is unambiguously an alert-pass read: its single caller is
:2921, which feeds the result straight into theAlertServerSnapshotpassed toengine.EvaluateServerAsyncon the next line.Why #2882's pin reports clean over it
Darling/Darling.Tests/AlertPassCommandTimeoutTests.cs:55scopes the whole scan to an allowlist of six filenames:DarlingWorker.csis not among them, so the site is never read.The pin already guards the direction that doesn't matter here. It asserts each of the six paths resolves, with a comment saying a renamed or moved alert-pass type must fail loudly "rather than silently shrinking the scan to the files that still resolve — an empty or partial sweep is how a guard starts reporting clean on code it no longer reads." That reasoning is right and the guard works: it catches a file leaving the list. There is nothing at all against a site that was never in it, which is the asymmetry that let this through.
Why an allowlist of filenames is the wrong scope, specifically
Group B established the general rule the hard way, twice in one PR, from opposite directions:
.Service's budget regimes do not respect file boundaries. It over-counted by attributing sites to a regime by file, and it under-counted by excludingDarlingObservability.WriteAnalysisStateAsyncbecause that site sits lexically inside a budgeted method while receiving the plainstoppingTokenrather than the budgeted one. The reliable test is:A filename allowlist encodes precisely the assumption that failed both times.
DarlingWorker.csis the clearest case in the project — it holds sites belonging to the startup regime, the sweep body, the analysis pass and the alert pass all at once.Shape of the fix
Both halves, in one change:
ORDER BY … LIMIT 1againstcpu_utilization_stats. Confirm the derivation rather than inheriting the constant by proximity.Whatever lands, the pin needs a red-first proof that it fails on this site being untimed — a widened scope that still can't see
DarlingWorker.cswould pass while guarding nothing, which is the same trap in a new coat.Not verified
The site's real-world duration was not measured; there is no reported failure attached to it.
cpu_utilization_statsis a hypertable and the query is a single-row read on a covered ordering, so the expectation is that it is fast and that this is latent exposure rather than an active problem — stated as an expectation, not a measurement.