Skip to content

PostgreSQL sessions family: an operator now learns when a pool near its ceiling is accepting more connections than this hour usually carries while completing no more work — the offered-vs-delivered co-fire reads the anomaly pair, and the raw TPS-trend hook is retired on the calibration's evidence (partial #3691, lane 18) - #3747

Merged
erikdarlingdata merged 1 commit into
devfrom
feat/3691-pg-offered-vs-delivered
Sep 19, 2026

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

What does this PR do?

Headline mechanism. Lane 18 of #3691 (design §3.6, D7) makes PG_CONNECTION_SATURATION's offered-vs-delivered co-fire real — and it is NOT the co-fire lane 3 sketched. A PostgreSQL operator whose pool is near its ceiling now learns whether the server is accepting more connections than this hour of the week usually carries while completing no more transactions than usual — arrivals queueing at the cliff rather than a busier application — and the saturation card is lifted ×1.3 when that is what the evidence says. PARTIAL for #3691 — do not close. Checklist line completed verbatim: Offered-vs-delivered co-fire.

The judgment: the raw-trend design was replaced by the anomaly pair on the calibration's evidence

Lane 3 parked a hook that would have amplified saturation when sessions climbed across the window while PG_TPS's in-window trend (tps_trend, second half of the window's rate minus the first half's) was flat or falling. The 2026-09-19 fleet calibration (measurements §A1) read fourteen days of pg_database_stats on the fifty PostgreSQL targets the product monitors and found the 5-minute TPS series has routine 20–50× bursts on every cluster — a per-server 5-minute maximum median of 1,710 against a p50 of 31. A first-half-vs-second-half trend of that series is noise: the hook would have amplified saturation on every batch job's shoulder, and missed real queueing at low absolute TPS. That design is retired here, not merely left parked (the retired block in PgTargetScorer.Sessions.cs says so and says why, so nobody resurrects it).

The honest offered-vs-delivered signal on this data is the pair of hour-of-week anomalies lane 9 built: ANOMALY_PG_SESSION_SPIKE fired (more connections than this hour usually carries — the offer rose) while ANOMALY_PG_TPS did not (no more work than usual was delivered). Both fired is a load surge — the anomaly family's own corroboration already handles it — and the arm stays quiet. A stock target with no baseline yet has neither anomaly and the arm is inert.

The change

PgTargetScorer.Sessions.cs. SaturationAmplifiers gains a third arm (OfferedVsDeliveredBoost, ×1.3) whose predicate is OfferedVsDeliveredFired(facts, key) — the pattern lane 15's checkpoint trigger set (TheCheckpointTriggerAmplifier_ReadsTheAnomaly_NotTheContextFact): the anomaly source is in FactScorer.ScoreAll's lookup, so a sibling anomaly's BaseSeverity > 0 is readable exactly as a regular fact's. "Fired" is the same BaseSeverity > 0 every sibling predicate asks. The TPS side has one defensive clause: a fired TPS anomaly whose peak sits at or under its centre (ratio ≤ 1.0) does not count as "throughput high" — vacuous today (AnomalyGate fires only on peak − centre ≥ threshold × dispersion), written so a future two-sided TPS detector's DROP reads as offered-vs-delivered instead of silencing the arm.

This is the one amplifier whose predicate writes metadata, on purpose and stated in the doc: the verdict needs the fact SET, which the base-severity seam never sees, and the predicate is the only seam with both the set and the fact. It stamps offered_vs_delivered (0 / 1), session_spike_ratio and tps_anomaly_ratio on every fired saturation fact — so get_analysis_facts shows the verdict either way — and nothing on a base-0 fact the amplifier pass never reaches. The stamp is a pure function of the set; evaluating it twice writes the same values. The ratio is AnomalyPeakOverCentre: peak over the bucket's robust centre (median when the bucket carries one, else mean — lane 15's baseline_ratio convention, so the "N× this hour's norm" and the "σ above the median" prose are one number), 0 on a zero centre (a metadata double must stay finite; the finding is persisted as JSON).

Lineage. The multiple is unmeasured and says so ("chosen, not measured — calibrate against analysis_findings co-fire rates"); a boost is not a bar the base was graded on, so the saturation fact's threshold_lineage stays 1 as lane L set it — the same rule ConnectionSaturationCoFireBoost rests on. Why 0.3 and not the siblings' 0.25: two baselined instruments agreeing; 1.0 × 1.3 = 1.3 leaves the second co-fire to carry a CRITICAL pool to the 1.5 notify line — one corroborator is WARNING, two page (#3584's rule). OfferedVsDeliveredBoost joins PgTargetMeasuredLineageTests.s_stillUnmeasured.

TpsTrendKey stays (PgTargetScorer.Database.cs, lane 6's file): the collector still stamps tps_trend on PG_TPS as a reader's figure and lane 6's PgTargetTempTests pins it; nothing in the sessions scorer reads it any more (pinned by source, comments stripped).

PgTargetAdvice.Sessions.cs. When offered_vs_delivered = 1, ONE sentence in the saturation investigation, value-stated from the metadata: "Connections rose to 4.3× this hour of the week's norm while throughput stayed at its norm — arrivals are queueing, not working: the server is accepting more connections than usual and completing no more transactions than usual, which is the shape of a pool at its cliff rather than a busier application." With no computable session ratio (a zero-centred bucket) it says "a level above" rather than inventing a multiple. Absent the stamp the block is byte-identical to lane 3's (pinned: composed from a stamped-0 fact and from an un-stamped fact, one string). The static block never carries it.

Co-fire gating, stated

  • Pair (session anomaly fired, no TPS anomaly, or a fired TPS anomaly at or under its centre) → fires, stamps 1.
  • Both fired → quiet, stamps 0 with both ratios.
  • TPS alone, no anomaly, or a session anomaly PRESENT but base 0 → quiet, stamps 0.
  • No saturation fact in the set → false, stamps nothing, no throw.
  • The PG_TPS fact and its trend are never read: a steeply falling trend with no anomaly is quiet; a steeply rising one with the pair still fires.

Boundary extensions (each one line)

  • Darling/Darling.Tests/PgTargetBetweenWavesTests.cs — item 11's pin asserted the parked hook's exact text (tps.Metadata.GetValueOrDefault(TpsTrendKey) <= 0); that text is gone by design. Re-pinned as TheOfferedVsDeliveredArm_ReadsTheAnomalyPair_AndTheRetiredTrendHookReadsNothing: the sessions scorer names both anomaly keys by constant and reads TpsTrendKey / tps_trend nowhere (comments and strings stripped); the key itself still equals "tps_trend".
  • Darling/Darling.Tests/PgTargetMeasuredLineageTests.cs — OfferedVsDeliveredBoost joins the Sessions s_stillUnmeasured roster (lane 14's precedent).

What it does NOT do

No anomaly-detector, TPS-collector, root-file, FactScorer, PgTargetFactKeys or graph edit; no rung; no new key or metric name; no production read (the calibration numbers are the coordinator's published file). It does not enable the PG_IDLE_IN_TRANSACTION co-fire (lane 14's double-counting argument stands; that hook stays parked). It does not touch lane 3's two v1 arms — their order, boosts and predicates are unchanged, and the pins that read AmplifierResults[0..1] hold at the same values. It does not move the stock-target 1.30 anomaly ceiling (v3). A SQL Server target's pass is byte-identical by construction: the arm lives behind the pg_sessions source's amplifier switch and reads ANOMALY_PG_* keys no SQL Server pass emits. Two doc lines in other lanes' files now describe a hook that no longer exists — reported below for the coordinator's shared-file batch, not edited here.

Test plan

Darling/Darling.Tests/PgTargetSessionsTests.cs (+4 unit pins, +1 gated e2e; 3 existing pins re-counted 2 → 3 arms with the third asserted unmatched): the third arm's boost, description and predicate through the real Amplifiers reflection seam — every gating row above, the stamps after each, the PG_TPS-trend blindness; AnomalyPeakOverCentre prefers the median, falls back to the mean, is 0 on a zero centre or a bare fact; through the REAL ScoreAll the pair lifts 1.0 to 1.55 (threshold_lineage still 1), both-fired leaves 1.25, and a saturation pass with NO anomaly is byte-identical in severity (1.25), path, active edges and advice text, its metadata gaining exactly offered_vs_delivered, session_spike_ratio, tps_anomaly_ratio (all 0) beside threshold_lineage; the advice sentence by exact string (4.3× = 90 / 21), its position (after the parked-share sentence, before the captures sentence), its absence on stamp 0 / no stamp / the static block, and the "a level above" form on a zero ratio.

Gated e2e (DARLING_TEST_PG): two stock-stamped servers, 31 days of one-minute pg_session_states (20–22 sessions) and pg_database_stats (≈10 tps ± 1 ripple) in two bulk inserts each (lane 9's shape), a pg_server_config snapshot at 100 / 3, both climbing to 90 sessions (40 active, 30 idle in transaction) for the last 250 minutes. QUEUED (throughput held): the detector alone yields exactly one anomaly (sessions, peak 90); the real analyze_server anchored at the window's end yields the saturation card at 1.55 with the sentence and the parked-share sentence; get_analysis_facts shows offered_vs_delivered = 1, session_spike_ratio in [90/22, 90/20], tps_anomaly_ratio = 0, threshold_lineage = 1. SURGE (60 tps through the climb): both anomalies; the card at 1.25, no sentence; the stamps 0 / [4.09, 4.5] / [5, 7].

EXECUTED here, not only compiled: the lane-14 net10.0 harness (AssemblyName=Darling.Tests, 41 PgTarget census / hygiene files incl. PgTargetWriteTests and PgTargetBetweenWavesV2Tests) copied under the worktree's test bin so the walk-up finds the tree, against a throwaway timescale/timescaledb:2.28.1-pg18 on Docker context desktop-linux (a darling superuser role, migrated to the current ladder by the fixture), torn down after: 527 / 527 green on 98d52fe1 + this change, including both this lane's gated e2e and every existing gated e2e in the set, DocCommentHygieneTests, LiveCleanupConversionRatchetTests, LivePostgresCollectionHygieneTests, PgTargetThresholdLineageTests, PgTargetMeasuredLineageTests, PgTargetSharedSwitchRoutingTests, PgTargetFactCollectorTests, StoreSqlClockDisciplineTests. Every number asserted was read off the executed run. Rebased onto 0caf572a9 (two unrelated commits) before opening; the lineage / between-waves / sessions classes re-run green after the roster edit.

Builds, zero warnings of ours: PerformanceMonitor.Analysis, PerformanceMonitor.Darling.Service, Darling.Tests, Lite.Tests, deprecated/Dashboard.Tests (-p:EnableWindowsTargeting=true on the test projects). The one Lite.Tests warning (LogTailOverlapThresholdPinTests.cs:95 xUnit2000) is pre-existing on dev. First execution of everything outside the harness set is CI's Windows job.

Which component(s) does this affect?

  • Lite
  • Darling
  • Lite Tests
  • Darling Tests
  • SQL collection scripts
  • Documentation
  • Full Dashboard (deprecated)
  • CLI Installer (deprecated)

How was this tested?

Above. No production store or fleet server was read.

Checklist

  • I have read the contributing guide
  • My code builds with zero warnings (dotnet build -c Debug)
  • I have tested my changes against at least one SQL Server version (the SQL Server pass is unreachable by the new arm; the routing pins that proved v1's byte-identity pass unchanged)
  • I have not introduced any hardcoded credentials or server names

Partial for #3691 (lane 18). CHANGELOG deliberately untouched — the coordinator splices this wave's entries. Not armed (review-dark window).

…maly pair, not a raw TPS trend (partial #3691)

Replaces the parked raw-trend hook on PG_CONNECTION_SATURATION with a third
amplifier that fires when ANOMALY_PG_SESSION_SPIKE fired and ANOMALY_PG_TPS did
not (x1.3, unmeasured), stamps offered_vs_delivered and both anomaly ratios on
the fact, and adds one value-stated sentence to the saturation advice. The
TpsTrendKey hook is retired on the 2026-09-19 calibration's evidence (A1).
@erikdarlingdata
erikdarlingdata merged commit 0a2e94c into dev Sep 19, 2026
6 of 7 checks passed
@erikdarlingdata
erikdarlingdata deleted the feat/3691-pg-offered-vs-delivered branch September 19, 2026 14:45
erikdarlingdata added a commit that referenced this pull request Sep 19, 2026
… PRs across the #3653, #3542 and #3691 waves, #3704, #3710, #3712 and #3735, spliced once (#3751)

Sixty-five PRs merged to dev since #3672 (the first splice) across two seats -
the coordinator's #3653 brains-review residue lanes and the wave-2 worker's
#3542 / #3691 PostgreSQL-target engine lanes - and by rule none touched this
file; each returned its entry to a buffer. This is the one splice. Text-only;
one file; nothing outside ## [Unreleased] and the definition block moves.

[Unreleased] gains a ### Changed between ### Added and ### Fixed (Keep-a-Changelog
order) for the nine entries whose subject is a designed behaviour that now behaves
differently - routing (#3712 confidence chooses the channel), gating (#3692 tempdb
persistence, #3734 blocking/LRQ gates, #3724 peak-and-mean), the wire shape
(#3719 one error envelope), the review guard's drift arm (#3673), lineage
metadata (#3723) and two grading changes (#3693, #3747). Twenty-two new
capabilities append to ### Added (the sixteen PostgreSQL-target fact families and
rungs, CONFIG_CHANGED, the recurrence label, the V134/V135 columns, the review
traps file, the measurement census); thirty-two defects append to ### Fixed.
Order within each section is ascending PR number, as #3672 did.

Every entry is one line and verbatim from its buffer (trailing whitespace only).
Four PR pairs fold into one entry as the buffers wrote them (#3738+#3746,
#3683+#3686, #3681+#3685, #3726+#3729); #3695 carries three entries. #3673 has
no issue and cites none, like the file's ninety-six other ref-less entries. Five
definitions added ([#3691], [#3704], [#3710], [#3712], [#3735]) at the end of the
[Unreleased] run, which stays one ascending run (3517..3735). No archive touched;
the split was not run.

tools/changelog/changelog_archive.py verify: all PASS (1,483 bold entries, floor
1,329; 1,363 distinct refs resolve; 1,372 definitions; CRLF throughout; 409,263
bytes under the ceiling). Client-hygiene grep over the file: zero hits.
erikdarlingdata added a commit that referenced this pull request Sep 19, 2026
…ne setting, so a New York target's Tuesday-09:00 bucket is 09:00 in New York in every season (partial #3691, lane 20) (#3757)

The clock read in PgBaselineProvider becomes a protected virtual seam (SQL Server body unchanged, byte for byte); PgTargetBaselineProvider overrides it once to read the TimeZone row of the latest pg_server_config snapshot at or before the window end, session-scoped sources excluded (the config facts' rule, copied), and hands the shared resolver the zone name. UTC is now the fallback for a target with no server-scoped snapshot, not the rule for the engine.

Doc riders: the root doc names the $4..$6 bind the base has made since #3749 and states the setting-then-UTC rule; the TpsTrendKey prose in PgTargetScorer.Database.cs and PgTargetFactCollector.Database.cs no longer claims a co-fire reads the trend (nothing has since #3747).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant