Repository navigation
Write the database_name column three collectors declared but never wrote - #2622
Conversation
pg_extension_availability, pg_column_stats and pg_index_bloat all gained a database_name COLUMN in v95 (#2599). I updated the SQL, the row record and the readers, and missed WritePayload in all three - so every row they produce is rejected at EndPayload with "wrote 6 payload values but declares 7 payload columns". The runtime check existed and was not enough. It only fires when the collector RUNS; all three are daily, and on Aurora two of them return zero rows for unrelated reasons - a missing grant and a missing extension. So the store had nothing from them, which is indistinguishable from a quiet server. It survived three schema versions that way. Found by pointing the service at a self-hosted PostgreSQL with 40 tables and 160 indexes, which is the first time these collectors had real data to write. Now pinned at build time across the whole catalog: a counting writer runs every collector's WritePayload against a default row and asserts the write count equals the declared column count. Collectors whose default row is not valid - the delta-based SQL Server ones dereference fields it leaves null - count as uncovered rather than as failures, and a coverage assertion keeps that from quietly swallowing the catalog. 50 of 66 exercised today. The columns and the writes are two halves of one statement about the table, and nothing but a test holds them together. Fixes the v95 regression in #2599. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
|
||
| /* The guard is only worth having if it actually covered the catalog. */ | ||
| Assert.True( | ||
| skipped.Count < CollectorCatalog.All.Count / 4, |
There was a problem hiding this comment.
This coverage threshold looks like it's sitting exactly on its own failure boundary.
CollectorCatalog.All.Count is 66 today. Integer division makes 66 / 4 == 16. The PR description states this probe currently exercises "50 of 66" collectors — i.e. skipped.Count == 16. That makes the assertion 16 < 16, which is false.
If that count is accurate, this test fails the moment it runs, on this very PR. Either:
- the "50 of 66" figure in the PR description is stale/approximate and the real skip count is ≤15 (worth double-checking against the actual CI run rather than trusting the prose), or
- the boundary needs to be
<=(or a less exact ratio), since<a truncated-integer-division threshold is fragile — the next collector added to the catalog can flip this pass/fail with no code change to the guard itself, purely fromCount / 4re-truncating.
Worth confirming against the real CI numbers before merge; a coverage guard that flakes on its own introduction undermines the "no silent gaps" goal it's meant to enforce.
|
Reviewed the diff. Summary: Core fix ( Lite/Darling parity — no drift. One thing worth checking before merge — left as an inline comment on No missing-index DMV concerns, no security/injection concerns — this PR only reorders/adds |
Found by standing up a self-hosted PostgreSQL rig — 40 tables, 160 indexes, all five instrument extensions loaded — and pointing the actual service at it. First time these collectors have had real data to write.
The bug
pg_extension_availability,pg_column_statsandpg_index_bloatall gained adatabase_namecolumn in v95 (#2599). I updated the SQL, the row record and the readers, and missedWritePayloadin all three:Every row they produce is rejected at
EndPayload.Why the existing runtime check wasn't enough
It only fires when a collector runs. All three are daily, and on the Aurora fleet two of them return zero rows for unrelated reasons — a missing grant on
pg_stats, a missing extension forpgstatindex. So the store simply had nothing from them, which is indistinguishable from a quiet server.It survived three schema versions that way.
pg_extension_availabilitywas the one collector actually broken on the fleet, and its last successful run predated the upgrade that broke it — so even the dogfood pass this morning saw a stale SUCCESS and moved on.Pinned at build time now
A counting writer runs every collector's
WritePayloadagainst a default row and asserts the write count equals the declared column count. 50 of 66 exercised, all matching.Collectors whose default row isn't valid — the delta-based SQL Server ones dereference fields it leaves null — count as uncovered rather than as failures, and a coverage assertion keeps that from quietly swallowing the catalog. A guard that skips most of what it claims to check is worse than none.
The columns and the writes are two halves of one statement about the table, and nothing but a test holds them together.
Worth noting
This is the second defect in two days that only a real instance could surface — after #2617, where a collector had never returned a row because I bounded each index and not the cycle. Both were invisible to CI, to a two-table container, and to the Aurora fleet. The rig is cheap and I'd keep it.