Repository navigation
The rest of the trend family buckets to a point budget: wait, CPU, tempdb, memory, perfmon, PG query duration (#3960) - #4007
Merged
Conversation
… PostgreSQL query duration) buckets to a point budget, the same way #3968 bucketed the first five (#3960) Reuses TrendBuckets/TrendBudget/TrendPayloads from #3968 rather than inventing a second mechanism. Gauges (memory, tempdb, CPU) publish the bucket's average with peak_* beside it; get_wait_trend and get_perfmon_trend's rate points sum work over summed seconds, never averaging per-collection rates. get_memory_trend's grant join moves from nearest-snapshot-within-30-seconds to same-bucket, averaging every grant snapshot in a memory point's bucket with the largest kept as peak_granted_mb. Extends the #3968-era test infrastructure for the new tools: TrendBucketsTests' cap sweep, TrendPayloadBudgetLiveTests' payload census (seeds and measures all six), and a new TrendFamilyBucketingLiveTests with hand-verified bucket math for each read's distinctive shape (wait's time-weighting, CPU's unclamped own-clock bucketing, tempdb's averaged-space/busiest-session split, perfmon's three-way interval-completeness shaping, the PG duration pair's recomputed mean). Fixes fallout in existing census/inventory tests from centralizing per-tool JSON building into the shared TrendPayloads builders: DarlingPgReadSqlParsesLiveTests (a shared SQL fragment reflected as a standalone query), ConsumedTimestampFrameDisciplineTests (a de-skewed site moving out of its per-SKU file), PerfmonCounterTypeRungTests and DarlingCustomViewsTests (bucket_minutes changing a payload shape a fixed-count assertion assumed), and their Lite twins. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdmA4ND1wLSqA91ax1m4xv
erikdarlingdata
enabled auto-merge (squash)
September 23, 2026 07:57
This was referenced Sep 23, 2026
erikdarlingdata
added a commit
that referenced
this pull request
Sep 23, 2026
…ra thread-id race, midnight UTC in Darling's daily summary, and the shard class-list parse (#4024) * The memory-trend grant-join tests pin their seeds past a bucket floor, so a run in a minute's last seconds no longer splits a snapshot from its sample #4007 put get_memory_trend on clock-aligned buckets. Two tests here still seeded at UtcNow minus N minutes with 3-6 second offsets, so about one run in ten crossed a bucket boundary (seen on #4022's Lite shard). Both now start 10 seconds past a five-minute floor, as the Darling twin and this file's third test already do. Red-watched: a 55-second pin reproduces the CI failure exactly. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdmA4ND1wLSqA91ax1m4xv * Two more CI flakes: the Entra thread check reads the caller's id before its await, and Darling's daily-summary test seeds today's run on today after midnight UTC - EntraDeviceCodeTests.TheAcquisitionIdentityReachesWhatTheAcquisitionAwaits read Environment.CurrentManagedThreadId after awaiting a LongRunning task with ConfigureAwait(false). The continuation can resume on that dedicated thread, so the check compared the thread with itself (dev CI, 06:18Z). - McpFilterSemanticsLivePostgresTests.DailySummary_StopsPaintingPurgedDaysGreen_AndPublishesTheHorizon seeded today's run two minutes back, which lands on yesterday in the first two minutes after midnight UTC; it failed beside its Lite twin at about 00:01Z. Lite got the guard in #3963; this mirrors it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdmA4ND1wLSqA91ax1m4xv * The test shards parse only the runner's JSON line of the class listing, so a dotnet CLI notice on stdout no longer fails a shard before it runs A dev push at 10:55Z failed Darling PG tests (2) in ConvertFrom-Json: "Additional text encountered after finished reading JSON content: W", line 2, i.e. text the dotnet CLI printed after the runner's one-line array. Both shard steps (Darling, Lite) now keep only the line that starts with "[" and throw, listing what came back, if there is not exactly one. Checked locally: the old parse reproduces the CI error on that shape; the new parse gives the same 401 Lite classes on the real listing. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdmA4ND1wLSqA91ax1m4xv --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
erikdarlingdata
added a commit
that referenced
this pull request
Sep 23, 2026
…ntries in their sections (#4080) Adds 42 entries and 42 link refs (#3992, #3995, #3996, #3998, #4001, #4002, #4003, #4007, #4010, #4011, #4013, #4015, #4020, #4022, #4025, #4029, #4030, #4031, #4036, #4038, #4039, #4040, #4044, #4047, #4048, #4049, #4050, #4051, #4055, #4061, #4063, #4064, #4065, #4066, #4067, #4068, #4069, #4070, #4071, #4073, #4074, #4078). Each PR's entry was buffered, and this lands every entry whose PR was merged on origin/dev when it ran. #3989 left 26 entries under bare 'Changed' and 'Fixed' lines above '### Added'. They move into '### Changed' and '### Fixed', below the new entries, and one blank line stays under [Unreleased]. Claude-Session: https://claude.ai/code/session_01Ua31ugERL5DmhFVRtf6keQ Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3960.
Why
#3968 bucketed five trend reads (
get_file_io_trend,get_lock_wait_trend, the duration pair,get_pg_io_trend,get_pg_database_trend) so their size stopped scaling with window x collection cadence. The rest of the family still returned every collection: measured on DARLING01 against the nightly build, a week ofget_tempdb_trendwas 1,965 KB,get_memory_trend1,481 KB,get_wait_trend1,051 KB,get_perfmon_trend852 KB, andget_pg_query_duration_trend160 KB.What changes
Extends the SAME mechanism (
TrendBuckets,TrendBudget,TrendPayloadsinPerformanceMonitor.Common) toget_wait_trend,get_cpu_utilization,get_tempdb_trend,get_memory_trend,get_perfmon_trend(both SKUs) andget_pg_query_duration_trend(Darling only) - no second mechanism invented.peak_*beside it, never a sum.get_wait_trendandget_perfmon_trend's rate points sum work over summed seconds (never an average of per-collection rates), with the busiest single collection kept as the peak.get_memory_trend's grant join (get_memory_trend: join the grants series so total_granted_mb carries real data (complete fix for #3529) #3548) moves from nearest-snapshot-within-30-seconds to same-bucket: both series bucket at the same width and a memory point averages every grant snapshot inside its bucket, with the largest kept aspeak_granted_mb. A bucket no snapshot falls into is null, never a fabricated 0.get_perfmon_trendshapes by the counter's kind (DeltaSeriesShaping.BasisFor): a level averages/peaks, a rate sums its deltas per interval-completeness class (rated, unknowable-zero, unrecorded-null) and keepspeak_per_second.get_pg_query_duration_trendrecomputes its mean from the bucket's summed calls and time (never averages per-interval means), keepspeak_mean_exec_ms, and reads its window'sfirst_mean_exec_ms/last_mean_exec_msfrom the served buckets' own carried fields./api/readpanels takeTrendBudget.Chartand bindbucket_minutes, matching The trend tools answer in time buckets sized to the window: a day of get_file_io_trend is 34 KB, not 1.4 MB (#3897) #3968's five.Ported by hand from an earlier lane's uncommitted work in the
3897-trend-tool-pointsworktree (reconciled against #3968 as merged - that worktree's base pre-dated the merge) plus this lane's own additions.Payload sizes, 168h default (auto-bucketed) vs before
get_tempdb_trendget_memory_trendget_wait_trendget_perfmon_trendget_pg_query_duration_trendget_cpu_utilizationMeasured live against
TrendPayloadBudgetLiveTests' fixture (a week of one-minute collections), not extrapolated.Fixed along the way
Centralizing per-tool JSON building into the shared
TrendPayloadsbuilders moved several call sites the existing census/inventory tests keyed on by file or by exact text. Each is a real site move, not a behavior regression - fixed in the same commit rather than filed:DarlingPgReadSqlParsesLiveTests: the newQueryDurationSnapshotsCtefragment (shared between the raw and bucketed PG duration SQL) contains a bareSELECTand was reflected as a standalone statement, which does not parse on its own. Turned it from aconst stringfield into a private method - invisible to the field-reflection this test uses - so the two real, complete statements that compose it stay in the parse-checked population and the fragment does not enter it directly.ConsumedTimestampFrameDisciplineTests(both SKUs):get_cpu_utilization'ssample_timede-skew is unchanged in SQL, but its JSON emission moved from an inline per-SKU projection into the sharedTrendPayloads.CpuUtilizationbuilder inPerformanceMonitor.Common, outside this census'sDarling/- andLite/-rooted file globs. Removed the two now-unreachable pinned rows (Darling'sDeSkewedAtReadentry, Lite'sDeclinedAmbiguousMcpSitesentry) with a comment explaining the site moved rather than regressed, and decremented the ratchet counts.PerfmonCounterTypeRungTests,PerfmonCounterTypeTests,McpDescriptionTruthPinTests,PerfmonIntervalAggregationTests: pinned an inline projection or exact description text that the shared builder move changed the shape or location of. Updated to check the shared builder call site / constant instead of a per-SKU copy.DarlingCustomViewsTests:get_wait_trend's wire parameter list gainedbucket_minutes.DarlingMemoryTrendGrantJoinTests/ Lite'sMemoryTrendGrantJoinToolTests: rewrote the grant-join tests for the bucket join (addedbucket_minutes: 1where the test needs per-collection granularity, replaced the now-meaningless "30-second boundary" test with a wider-bucket average/peak test).Test plan
Darling.Testsfull run against a live rig: 12,957 total, 0 failed, 29 skipped (unrelated live-cloud gates).Lite.Testsfull run: 5,161 total, 0 failed.TrendFamilyBucketingLiveTests(Darling): hand-verified bucket math for each read's distinctive shape - wait's time-weighting and dropped-unrated-bucket, CPU's unclamped own-clock bucketing, tempdb's averaged-space-but-busiest-session split, perfmon's three-way interval-completeness shaping (rate and gauge), the PG duration pair's recomputed mean with peak/first/last.TrendPayloadBudgetLiveTestsextended: seeds and measures all six new/extended tools at their default and at-cap widths, plus refusal checks one bucket-minute narrower than the cap.TrendBucketsTests' cap-refusal sweep extended with the six newTrendBuckets.*MaxPointsconstants.DarlingWebEndpoints.cs,server-tabs.js) takes the chart budget for the new tools, matching The trend tools answer in time buckets sized to the window: a day of get_file_io_trend is 34 KB, not 1.4 MB (#3897) #3968's five.Coordinator note
This closes out the
3897-trend-tool-pointsworktree: its dirty patch applied cleanly against freshdev(post-#3968) with no superseded hunks, and everything in it plus this lane's own additions are in this PR. Safe to remove that worktree.🤖 Generated with Claude Code
https://claude.ai/code/session_01GdmA4ND1wLSqA91ax1m4xv