Repository navigation
get_pg_io_trend's automatic subject choice reads each series' endpoints on TimescaleDB, not every row in order (#3961) - #4003
Merged
Conversation
…ts on TimescaleDB, not every row in order (#3961) DominantIoSubjectSql sorts every pg_io_stats row in the window before its LAG window function can rank them: 3.8-5.8 s over 168 h on DARLING01. The ranking only needs each series' summed increase, and for a counter that never rewinds that sum is its last value minus its first, answered from a HashAggregate in 0.30 s on the same window with an identical top-8. DominantIoSubjectEndpointsSql reads the fast way on a TimescaleDB store (first()/last() are TimescaleDB's) and flags any series that shows a sign of rewinding - a moved stats_reset, a partly-NULL counter, or a counter below its first value or below its peak - so the reader falls back to the exact LAG form for that window instead of trusting an endpoint reading that could be wrong. A plain-PostgreSQL store keeps the LAG form outright, gated by IsTimescaleDbStoreAsync. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdmA4ND1wLSqA91ax1m4xv
erikdarlingdata
enabled auto-merge (squash)
September 23, 2026 06:53
erikdarlingdata
added a commit
that referenced
this pull request
Sep 23, 2026
…ntries in their sections (#4080) Adds 42 entries and 42 link refs (#3992, #3995, #3996, #3998, #4001, #4002, #4003, #4007, #4010, #4011, #4013, #4015, #4020, #4022, #4025, #4029, #4030, #4031, #4036, #4038, #4039, #4040, #4044, #4047, #4048, #4049, #4050, #4051, #4055, #4061, #4063, #4064, #4065, #4066, #4067, #4068, #4069, #4070, #4071, #4073, #4074, #4078). Each PR's entry was buffered, and this lands every entry whose PR was merged on origin/dev when it ran. #3989 left 26 entries under bare 'Changed' and 'Fixed' lines above '### Added'. They move into '### Changed' and '### Fixed', below the new entries, and one blank line stays under [Unreleased]. Claude-Session: https://claude.ai/code/session_01Ua31ugERL5DmhFVRtf6keQ Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3961.
Why
get_pg_io_trend's automatic subject choice (DominantIoSubjectSql) sorts everypg_io_statsrow of thewindow before its
LAGwindow function can rank the (backend_type, object_type, context) series: onDARLING01 over 168 h that was measured at 3,767-5,791 ms (per Erik's issue comment, re-measured deciding
whether to fold this into #3897/#3968 - it was not, because the fix needs a ruling on trusting endpoints).
The trend it feeds answers in 269 ms on the same window.
What changes
DarlingPgTrendReader.DominantIoSubjectEndpointsSql: ranks each series on `last(x, collection_time)(aHashAggregate, no sort) instead of differencing every row, and reportsany_rewound- true when any series in the window shows a sign it went backwards (stats_resetmoved, a counter is NULL on some rows but not others, or a counter dipped below its first value or ended below its peak). Its HAVING/ORDER BY tail is byte-identical toDominantIoSubjectSql`'s (pinned by test) so the twocan never rank the same sums differently.
GetDominantIoSubjectAsynctakes a newfromEndpointsparameter: true tries the endpoints read first andonly falls through to the exact LAG form when
any_rewoundsays a reset happened in this window, or theendpoints read found no qualifying pair at all.
first()/last()are TimescaleDB hyperfunctions, so aplain-PostgreSQL store (Store-object convergence never re-runs on a plain-PostgreSQL store: the hourly pass is gated on TimescaleDB, so the Ungated and Tuning steps heal only at restart #3913) always keeps the LAG form.
DarlingPgTrendReader.IsTimescaleDbStoreAsync, gatingfromEndpointsat the one call site inDarlingMcpPgTrendTools.GetPgIoTrend- the samepg_extensioncheckTimescaleSupport.DetectAsyncuseson a live connection, asked here of a data source. Cheap enough (one indexed catalog lookup) to ask fresh
per call rather than memoize, unlike the window-scanning reads it gates.
is hand-checked by the LAG form itself.
Diagnosis, confirmed on my rig
Erik's own comment on the issue already measured this on DARLING01 (real production scale, 788,341 rows,
8 chunks / 7 compressed): LAG form 3,767-5,791 ms, endpoints form 304 ms, identical top-8 pairs, no
stats_resetin that window - and flagged the two open questions this PR resolves: the reset-detectionrule (a ruling, since the shipped comment on the LAG query argues deliberately about resets) and
first()/last()being TimescaleDB-only (routing needed).On my own rig (60,480 synthetic
pg_io_statsrows, 7 daily chunks, 6 of 7 compressed - the same shape,smaller scale): LAG form 62.6 ms, endpoints form 52.9 ms, identical chosen pair. The relative win is smaller
here because the win scales with rows-per-chunk (sort cost is worse than linear) and my rig has about 13x
fewer rows per chunk than DARLING01 - direction and mechanism confirmed, magnitude needs DARLING01's scale
to reproduce exactly.
Lite: no equivalent. Lite has no PostgreSQL targets and no
pg_io_statstable or "dominant subject"auto-choice pattern of any kind (grepped
Lite/forpg_io_stats,Dominant,first(...collection_time),last(...collection_time)- no matches). Nothing to port there.Test plan
PgTrendReaderTests.TheEndpointChoice_RanksTheLagFormsSums_AndFlagsAWindowThatRewound- unit, pins theendpoints SQL's shape (HAVING/ORDER BY tail identical to the LAG form's, filters identical, every
counter's rewind check present, no
LAG(in it) andTimescaleExtensionPresentSql's text.TrendBucketingLiveTests.PgIoSubjectChoice_FromEndpoints_IsTheLagFormsChoice_AndARewoundWindowIsHandedToIt-live, three pairs over five snapshots where trusting the endpoints blindly would pick the wrong
subject (an autovacuum-worker reset drops its sum from 5,100 to 100, which would hand the default to
a client-backend series that never rewound); proves
fromEndpoints: true/falseagree everywhere,any_rewoundfires correctly (including a rewind that never movesstats_reset), half a subject stillconstrains the endpoint read,
IsTimescaleDbStoreAsyncreads true on a real TimescaleDB store, and anend-to-end call through
GetPgIoTrenditself resolves the same subject. Proved to fail on the old(rewind-blind) shape by temporarily dropping the
any_rewoundcheck and re-running - it failedexactly as expected, then the sabotage was reverted.
*PgIo*,*PgTrend*,*DarlingMcpPgTrendTools*,*McpPayloadContractCensusTests*,*DocCommentHygiene*,*LivePostgresCollectionHygieneTests*,*StorageCommandTimeoutTests*- allgreen (the timeout gate caught a real miss: my new
IsTimescaleDbStoreAsynccommand needed anexplicit
CommandTimeout, now fixed).Darling.Tests,DARLING_TEST_PGset - 12,951 total, 0 failed, 29 skipped (unrelated liveSQL Server E2E tests with no
DARLING_TEST_SQL). One run surfaced a genuine, pre-existing flake: mynew live test originally shared its time anchor and (backend_type, object_type, context) key with an
existing
pg_io_statsfact in the same file/server_id, and running concurrently with it in the fullsuite (not just in isolation) cross-contaminated both. Fixed by moving my test's window to a date none
of this file's other facts touch. Re-ran the full suite after the fix: clean.
Lite.Testsnot run (no Lite file touched, no shared project touched).🤖 Generated with Claude Code
https://claude.ai/code/session_01GdmA4ND1wLSqA91ax1m4xv