Repository navigation
plan_correction has the same bimodal-cost BODY_OVERRUN risk #2701 fixed for query_store, and was never included #2717
Description
Background
Erik flagged plan_correction's Collector Cost Regression alert on one fleet server (217,072 ms/day, 3.4x its 14-day baseline of 63,758 ms/day) as unacceptable and asked for further performance tuning beyond #2687 (materialize sys.dm_db_tuning_recommendations to #temp before joining Query Store catalog views) and #2688 (TRY_CONVERT→TRY_CAST).
Investigation
Confirmed via get_collector_cost: fleet-wide plan_correction total daily cost is trending DOWN since #2687/#2688 shipped (4.7M → 9.4M → 7.6M → 5.7M-partial-day ms across the ~40-server fleet) — those fixes are real and working.
The one affected server is the outlier. get_collection_health shows plan_correction there: avg_duration_ms 1057, p95 1554, max_duration_ms 21351, with fanout showing 97% of the worst cycle's cost concentrated in one database. That server's other databases are trivial by comparison.
Checked collect.query_store_plan_map for that database directly: 36,232 distinct plan_ids, 28,260 distinct digests — the identical application-class decimal-parameter-instability signature already root-caused for query_store on other servers earlier tonight. Several of the highest-volume plan_hash groups do collapse to a single digest (meaning many plan_ids there genuinely do share identical content, unlike the earlier-found servers' top offenders which showed zero collapsing), so this server's shape isn't byte-identical to the others, but the scale and mechanism are the same: an abnormally large, still-churning distinct-plan population feeding an Automatic Plan Correction recommendation set (200 live recommendations, all against the one large database, confirmed static in count across three checks tonight) that plan_correction must correlate every cycle.
plan_correction's own SQL has no further tuning headroom. It's already correctly seek-based post-#2687. The occasional 20+ second spike is proportional to real catalog size on an already-affected database — a data-volume problem downstream of the same app-side bug already being fixed on Erik's end, not a query-shape defect.
The actual actionable finding
plan_correction's cost shape — average ~1 second, occasional 20-230+ second spike depending on the affected server — is exactly the bimodal shape #2701 fixed for query_store (average 5-35s, occasional 100-230s spike on the same class of affected server), which caused query_store to block every other due collector sharing its per-server sequential body and risk BODY_OVERRUN. #2701's detach fix (RunDetachedQueryStoreAsync, IsQueryStoreCollector gate) applies only to query_store. plan_correction is still awaited inline in RunDueCollectorsAsync's sequential foreach, so a plan_correction spike on an affected server can still block that server's other due collectors for the duration, and — combined with query_store's own occasional spike on the SAME server — can still push a cycle toward BODY_OVERRUN.
plan_correction's own PerItemWallClockBudget of 120 seconds (set in #2673, "this collector cannot run minutes on a monitored server even if a future shape regresses") already anticipates runs heavy enough to risk overlapping its own next scheduled tick — the same self-overlap concern #2165's QueryStoreServerGate exists to prevent for query_store.
Proposed fix
Generalize #2701's detach pattern rather than special-casing a second collector: a small per-(server, collector) single-flight gate, structurally like QueryStoreServerGate but collector-agnostic, so plan_correction (and any future collector shown to have this same bimodal shape) can be detached from the sequential body and single-flighted against itself without inline-blocking every other due collector on an affected server. QueryStoreServerGate itself stays untouched (it has an orthogonal, second job: coordinating against the separate first-contact backfill loop, which a generic detach gate doesn't need to solve).
Activity
Detach plan_correction from the sequential body, generalizing #2701 (#…
erikdarlingdata commentedon Sep 1, 2026
Fixed by #2720 (plan_correction detached from the sequential body, generalizing #2701). Merged to
dev; ships in the next release.Pin the sweep-body detach policy and the invariant that makes it safe…