You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
PlanRegressionSql timeouts are silently swallowed — a cancelled query is indistinguishable from 'no plan regressions' #2826
When PlanRegressionSql is cancelled by its (unchosen, inherited) 30 s Npgsql deadline — 325 times on 2026-09-03 alone, see #2827 — the failure is discarded with no log line, no retry, and no fact. "The plan-regression query timed out" and "this server has no plan regressions" produce byte-identical output.
catch(Exceptionex)when(!AnalysisShutdown.IsExpectedAbandon(ex,context.CancellationToken)){/* Table may not exist or have no data. An abandonment is NOT swallowed here (#2443). */}
The body is empty. A PostgresException for 57014 query_canceled is neither "table may not exist" nor "no data", and nothing distinguishes it from the early return four lines above:
if(offenderCount==0)return;
Both paths emit no PLAN_REGRESSION fact. So on the servers where this query reliably exceeds 30 s, plan-regression detection is silently disabled, and has been for as long as the query has been over the line.
This is the same family as #2801 (abandoned cycle recorded SUCCESS), #2796 (cancelled watermark read returned null, indistinguishable from a first run, and silently downgraded the collection window), #2816 (probe:0ms misattributed to other:), #2818 (all 17 relations threw but every message was discarded by a null logger), and #2820 (timed-out baseline cached a null for an hour). In every case the code was correct about the happy path and blind about its own failure.
It is a category, not an instance
grep -c "Table may not exist or have no data" across Darling/ returns 25. Fixing only the plan-regression site would be scenario-shaped — the same repair that had to be made twice in #2822's recovery path and twice again in the PgStatementText / #2344 enumerated-pin gap.
The fix is cheaper than previously assessed
#2795 records that logging here "needs an ILogger on PgFactCollector, which today takes only an NpgsqlDataSource — a ctor change across 7 call sites."
That is not required. Three siblings in the same project already take the logger as an optional parameter, so no call site changes at all:
PgFactCollector(NpgsqlDataSource postgres) can take the same optional parameter and log in the catch, matching the convention already established beside it.
Suggested shape
Distinguish the three outcomes rather than collapsing them:
Expected abandon — already handled by the when filter, unchanged.
Anything else — log, rather than assuming "table may not exist".
Worth considering alongside: a fact or collection_log-visible marker so the gap is detectable from the store and not only from the app log, since the analysis pass leaves no collection_log row of its own.
A pin should assert over the family of catch sites by reflection rather than the one method touched — an enumerated pin is exactly how a broken sibling survived a green suite in #2344 and again in PgStatementText.
Resolved by #2832, merged to dev. Closing manually: Fixes #N only fires on the dev->main release merge, so completed work otherwise sits open until release.
Summary
When
PlanRegressionSqlis cancelled by its (unchosen, inherited) 30 s Npgsql deadline — 325 times on 2026-09-03 alone, see #2827 — the failure is discarded with no log line, no retry, and no fact. "The plan-regression query timed out" and "this server has no plan regressions" produce byte-identical output.Darling/PerformanceMonitor.Darling.Analysis/PgFactCollector.QueryPerf.cs:479:The body is empty. A
PostgresExceptionfor57014 query_canceledis neither "table may not exist" nor "no data", and nothing distinguishes it from the early return four lines above:Both paths emit no
PLAN_REGRESSIONfact. So on the servers where this query reliably exceeds 30 s, plan-regression detection is silently disabled, and has been for as long as the query has been over the line.This is the same family as #2801 (abandoned cycle recorded SUCCESS), #2796 (cancelled watermark read returned
null, indistinguishable from a first run, and silently downgraded the collection window), #2816 (probe:0msmisattributed toother:), #2818 (all 17 relations threw but every message was discarded by a null logger), and #2820 (timed-out baseline cached anullfor an hour). In every case the code was correct about the happy path and blind about its own failure.It is a category, not an instance
grep -c "Table may not exist or have no data"acrossDarling/returns 25. Fixing only the plan-regression site would be scenario-shaped — the same repair that had to be made twice in #2822's recovery path and twice again in thePgStatementText/ #2344 enumerated-pin gap.The fix is cheaper than previously assessed
#2795 records that logging here "needs an
ILoggeronPgFactCollector, which today takes only anNpgsqlDataSource— a ctor change across 7 call sites."That is not required. Three siblings in the same project already take the logger as an optional parameter, so no call site changes at all:
PgBaselineProvider(NpgsqlDataSource postgres, ILogger? logger = null)PgAnomalyDetector(NpgsqlDataSource postgres, PgBaselineProvider baselineProvider, ILogger? logger = null)PgPlanFetcher(…, ILogger? logger = null)PgFactCollector(NpgsqlDataSource postgres)can take the same optional parameter and log in the catch, matching the convention already established beside it.Suggested shape
Distinguish the three outcomes rather than collapsing them:
whenfilter, unchanged.PostgresException57014,NpgsqlExceptionwrapping a timeout) — log at WARN naming the collector and the server, so it is greppable the way[PgBaselineProvider] Baseline query for io_latency did not finish within its command timeoutis. That message is what made io_latency baseline times out: the robust scaffold's MAD tiers join on an OR'd non-equi predicate (23.7s -> 4.2s) #2820 findable at all.Worth considering alongside: a fact or
collection_log-visible marker so the gap is detectable from the store and not only from the app log, since the analysis pass leaves nocollection_logrow of its own.A pin should assert over the family of catch sites by reflection rather than the one method touched — an enumerated pin is exactly how a broken sibling survived a green suite in #2344 and again in
PgStatementText.