Repository navigation
Stage index_object_stats definition metadata instead of correlating per index (#3508) - #3510
Merged
Merged
Conversation
…er index (#3508) The per-database body staged the three stats DMVs into #temps (the #1135 fix) but built the per-index DEFINITION metadata in the final SELECT as five correlated subqueries against the live catalog, evaluated per index row: key_columns and included_columns (FOR XML over sys.index_columns/sys.columns), the two FK EXISTS over sys.foreign_key_columns, and the compression TOP(1) over sys.partitions. On a wide schema that is ~5 catalog probes per index, and the cost scales with index count - the body-cost half of #3477 (a 54K-index database spends ~28 s here). Extend the existing staging to the definition catalog, the technique sp_IndexCleanup already uses: stage sys.index_columns (+ column names, once, with a supporting clustered index), the FK-backed/FK-referenced key-column sets, and the per-index lowest partition compression into #temps, then LEFT JOIN them in the final SELECT with no correlated subqueries. The emitted key/include strings and every flag are byte-identical, so the Stage-2 dedupe and FK-protection logic are unaffected - a plan-shape change only. IndexObjectStatsCollector is shared, so Lite and Darling both get it. Measured on a synthetic 56,001-index database (empty tables, so this is the definition-metadata cost in isolation): the body drops from ~4.1 s to ~2.0 s, and an EXCEPT-both-ways parity check returns zero rows in either direction at 12K and 56K indexes. A guard test pins that the definition metadata is staged, not correlated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RzwoqcD62jKyp6tzsWpZLz
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3508. The body-cost half of #3477 (the width half is the per-collector DB scope; this is the per-database body).
Problem
index_object_statsstaged the three stats DMVs into#temps(the #1135 fix) but built the per-index definition metadata in the finalSELECTas five correlated subqueries against the live catalog, per index row:key_columns/included_columns(FOR XMLoversys.index_columns/sys.columns), two FKEXISTSoversys.foreign_key_columns, and the compressionTOP (1)oversys.partitions. ~5 catalog probes per index, scaling with index count — a 54K-index database spends ~28 s in this body.Fix
Extend the existing staging to the definition catalog — the technique
sp_IndexCleanupalready uses. Stagesys.index_columns(+ column names, once, with a supporting clustered index), the FK-backed / FK-referenced key-column sets, and the per-index lowest partition compression into#temps; thenLEFT JOINthem in the finalSELECTwith no correlated subqueries. Emitted strings and flags are byte-identical, so Stage-2 dedupe and FK-protection are unaffected — a plan-shape change only. The collector is shared, so Lite and Darling both get it.Measured
Synthetic database, 56,001 indexes on empty tables (so this isolates the definition-metadata cost):
EXCEPTboth ways returns 0 rows in either direction at both 12K and 56K indexes. Per-step instrumentation confirms the correlated per-index catalog probing is gone; the residual is the unchangeddm_db_index_operational_statsread plus the irreducible cost of building the delimited key/include strings. (Empty tables, so absolute times run below a real 54K-index instance whose operational-stats DMV is heavier; the ~2x and the parity are the transferable results.)Tests
DefinitionMetadata_IsStaged_NotCorrelatedPerIndexpins that the definition metadata is staged, not correlated (a revert drops the markers).NpgsqlRootCertificateValidationTests.ChainShape…) is the known machine-local TLS cert-chain test on a path this change never touches (it passes in CI on Linux).🤖 Generated with Claude Code
https://claude.ai/code/session_01RzwoqcD62jKyp6tzsWpZLz