Skip to content

Query Store: skip readable-secondary replicas (readonly_reason bit 8) — RDS/geo replicas were collecting the primary's replicated QS - #1558

Merged
erikdarlingdata merged 2 commits into
devfrom
feature/qs-skip-replica-readonly
Jul 18, 2026
Merged

erikdarlingdata merged 2 commits into
devfrom
feature/qs-skip-replica-readonly

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

Summary

Field evidence from the 24-server RDS fleet: every read replica was collecting query_store, each database showing actual_state = READ_ONLY, desired_state = READ_WRITE, readonly_reason = 8 — the engine's documented "read-only because this database is a readable secondary replica." A secondary's persisted Query Store tables are the primary's content arriving via replication — collecting there yields duplicate, lagged primary data (×24 copies of the same QS), and is exactly the pathological-volume shape that fed the #1557 memory incident.

Why both existing guards missed

  • The enumeration's replica guard joins sys.dm_hadr_database_replica_states — Always On AG only. An RDS read replica is not an AG member, so it passes the "not in any AG" arm looking like a standalone primary.
  • The Query Store: attribute runtime stats to their replica role; tighten the actual_state gate #1546 eligibility gate actual_state IN (1, 2, 4) deliberately admits READ_ONLY (correct for an operator-set read-only QS on a primary) but never consults why it is read-only.

Fix

AND readonly_reason & 8 = 0 (bitmask form) at all three per-database eligibility sites: the shared collector's on-prem and Azure enumeration queries (serving Lite + Darling) and the deprecated Dashboard's install/09 proc (bug-fix mirror). Bit 8 is the engine's mechanism-agnostic replica flag — AG secondaries, RDS read replicas, and geo-secondaries are skipped uniformly, with no dependence on which replication technology produced the secondary.

Deliberately still collected: operator-set read-only QS on a primary (readonly_reason <> 8), and READ_CAPTURE_SECONDARY (state 4 — a 2022+/2025 secondary capturing its own genuinely-local workload).

Testing

  • Both enumeration-text pins updated to assert the new clause (on-prem + Azure).
  • Full suites, Release: Lite.Tests 1369/0, Darling.Tests 2161/0.
  • install/09 change rides the standard SQL-version-matrix PR checks (2017/2019/2022/2025).

🤖 Generated with Claude Code

erikdarlingdata and others added 2 commits July 17, 2026 20:10
Field evidence: 24 RDS read replicas all collecting query_store, each
showing actual_state=READ_ONLY, desired=READ_WRITE, readonly_reason=8
(read-only BECAUSE readable secondary). A secondary's persisted QS
tables are the PRIMARY's content via replication - collecting there is
duplicate, lagged primary data x24, and exactly the pathological
volume shape that fed #1557.

Why both guards missed: the enumeration's HADR join only catches
Always On AG secondaries (RDS replicas pass the "not in any AG" arm),
and the #1546 actual_state IN (1,2,4) gate deliberately admits
READ_ONLY without asking WHY.

Fix: readonly_reason & 8 = 0 (bitmask) at all three eligibility
sites - shared collector on-prem + Azure enumerations (Lite+Darling)
and the deprecated Dashboard's install/09 proc. The engine's own
mechanism-agnostic replica flag covers AG, RDS, and geo-secondaries
uniformly. Still collected: operator-set read-only QS on a primary
(reason<>8) and 2022+/2025 READ_CAPTURE_SECONDARY (state 4, real
local secondary workload).

Pins updated: both enumeration-text tests assert the new clause.
Lite.Tests 1369/0, Darling.Tests 2161/0 (Release, full).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The rationale comment sat INSIDE the N'...' eligibility string and
contained an apostrophe, terminating the literal - all four
SQL-version-matrix checks failed on the proc definition. The clause
stays; the comment now lives in the static proc body where
apostrophes are legal, with a note saying exactly why.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant