Skip to content

The PostgreSQL analysis vocabulary absorbs what six content lanes reported back: a stable PG_BAD_ACTOR alias the graph resolves to the window's heaviest statement, the two vacuum-side config facts that were inert because no arm ever gave them a base, PostgreSQL 16's reserved_connections in the connection ceiling, and the bad actor's first next-read (#3542, between waves) - #3688

Merged
erikdarlingdata merged 1 commit into
devfrom
feat/3542-between-waves
Sep 19, 2026

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

What does this PR do?

Headline mechanism. Six content lanes of #3542 v1 (wave A: knobs, sessions, vacuum, waits, temp, queries, posture) each reported back one or more items they could not make inside their file-disjoint boundary — a key the shared vocabulary lacked, an arm that belonged in another lane's scorer file, a census whitelist that needed one admission. This PR is the between-waves batch of those items, decided by the wave owner and implemented here as one shared-file change so the next wave (lane 9 and the follow-ups) rebases onto one head rather than eleven. Partial for #3542 — do not close.

The lies it stops telling

  • Two pg_config facts were inert by construction. CONFIG_PG_AUTOVACUUM_OFF and CONFIG_PG_MAINT_WORK_MEM are emitted by the config snapshot read with source pg_config, so PgTargetScorer.ScoreConfigFact is the only switch that can give them a base — and it had no arm for either, so they fell to default: 0.0. FactScorer.ScoreAll skips amplifiers on a base of 0 (FactScorer.cs:100), so lane 4's VacuumConfigCoFireAmplifiers (+0.5 on the backlog having fired) could never lift them, and the vacuum chain's PG_AUTOVACUUM_BACKLOG → CONFIG_PG_MAINT_WORK_MEM edge never had a destination that fired. A target with autovacuum = off produced a config fact at 0 and no card. Now autovacuum = off scores 0.9 (engine-defined / posture — the setting is the evidence, design §3.1's band; the backlog co-fire lifts it to 1.35), and maintenance_work_mem is evidence-gated exactly as work_mem is (D5): the vacuum collector stamps the backlog ratio onto the knob fact off the in-memory list (the seam Database.cs uses for the spill rate), the arm returns the 0.4 advisory base when that ratio is at or past the table's own trigger line (1.0 — engine-defined) and 0 without the stamp; alone it roots nothing (not an advisory root), beside the fired backlog it reaches 0.6 and the edge fires.
  • The connection ceiling read high on PostgreSQL 16+. usable = max_connections − superuser_reserved_connections ignored the second carve-out PostgreSQL 16 added (reserved_connections, for members of pg_use_reserved_connections; default 0) — lane 3 noted it as "reads that many slots HIGH — noted, not compensated" because the name list was lane 2's file. The snapshot list now carries it, PgTargetFactKeys.ConfigReservedConnections is its context fact, the ceiling subtracts it when present and 0 when absent (a pre-16 snapshot has no such row and the fact is simply not emitted — the same arithmetic as the 16+ default), and the saturation advice states the third term only when it is non-zero, so every pre-16 sentence is byte-identical.
  • No edge could name a bad actor. RelationshipGraph.AddEdge takes an exact destination and lane 7's keys are dynamic (PG_BAD_ACTOR_<queryid>), so lanes 6 and 9's "edge INTO the statement" had no key to write against (lane 7 recorded the two options in PgTargetRelationshipGraph.Query.cs so the decision would be made once). It is made here: PgTargetFactKeys.BadActorFamily = "PG_BAD_ACTOR" is a stable ALIAS — never a fact's key, and it does not match BadActorKeyPrefix (no trailing underscore) so no prefix arm claims it — and PgTargetRelationshipGraph.GetActiveEdges resolves an alias destination, per call, to the highest-severity PG_BAD_ACTOR_* fact present (ties by ordinal key; none → the edge is dropped from the active set, never thrown). The rewrite returns a COPY of the edge; the stored edge keeps the alias, because the graph is a process-wide singleton walked for many servers.

The items, as decided

# Item (reporting lane) Decision Change Pin
1 Edge INTO a bad actor cannot name a dynamic key (7) Stable alias + story-build-time resolution in the PG graph PgTargetFactKeys.BadActorFamily; PgTargetRelationshipGraph.GetActiveEdges override + ResolveBadActor / IsBadActorAlias; one-word seam in RelationshipGraph.cs: GetActiveEdges becomes virtual (body byte-identical) Two candidates → higher severity; tie → ordinal; none → dropped, not thrown; through the real InferenceEngine.BuildStories the path is PG_TEMP_SPILL → PG_BAD_ACTOR_7; stored edge keeps the alias; non-alias edges pass through as the same instances; the SQL Server graph class does not override
2 ComparisonBanding.PlanCacheIdentityPrefix ≠ PG_BAD_ACTOR_ (7) Do NOT add it: queryid is stable within a major, so presence is the honest reading (the top-5 cut is the one churn source that remains) Comment only on the const IsPlanCacheIdentityKey(PG_BAD_ACTOR_42) false
3 CONFIG_PG_AUTOVACUUM_OFF base arm missing (4) Value > 0 ? 0.9 : 0, lineage engine-defined / posture AutovacuumOffPostureSeverity = 0.9 + arm in PgTargetScorer.Config.cs (owner: vacuum) 0.9 / 0; no threshold_lineage stamp; advisory root; ScoreAll alone 0.9, beside the fired backlog 1.35
4 FROM/JOIN census forbids collection_log (4) Admit it, one line PgTargetFactCollectorTests.AllSql_EveryFromJoinTarget_… whitelist + comment naming the xmin-horizon arm (#3537/#3642's honest denominator counts SUCCESS captures) The census IS the edit; nothing else admitted
5 PG_MONITORING_PERMISSIONS 0.5 vs 0.4 (3) No change — stays 0.5 — (lane 3's pins stand)
6 reserved_connections missing from the snapshot list (3) Add to the IN (…), a key, subtract in the ceiling PgTargetFactCollector.Config.cs name list + emitter; PgTargetFactKeys.ConfigReservedConnections; PgTargetFactCollector.Sessions.cs ceiling line (?.Value ?? 0) + reserved_connections metadata; PgTargetToolRecommendations row (the "every key has next reads" census); PgTargetAdvice.Sessions.cs: the arithmetic sentence states the third term only when > 0 Name in SQL; base 0; not an advisory root; source pin on the ceiling line and the mandatory-pair gate; advice byte-identical at 0 (90 / (100 − 3) = 93%), honest at 2 (90 / (100 − 3 − 2) = 95%)
7 PgTargetScorer.Config.cs is a four-lane file (4) Owner-family labels, no code move Class summary + /* ── owner: knobs (lane 2) ── */ / temp / vacuum / sessions section markers Text only; PgTargetTempTests' case PgTargetFactKeys.ConfigWorkMem: pin unchanged
8 Bad-actor next-reads lack get_pg_query_duration_trend (7) Add it FIRST (the advice's "first question") PgTargetToolRecommendations bad-actor arm Tool registered under that exact name (DarlingMcpPgTrendTools.cs:124); leads the list; advice prose names it; PgTargetQueriesTests pin moved to the three-tool order
9 numbackends on pg_database_stats (3) A RUNG → not this PR One line on #3542 (comment, after open) —
10 CONFIG_PG_MAINT_WORK_MEM inert for the same reason as item 3 (6, addendum) Evidence-gated, the work_mem shape MaintWorkMemBacklogRatioKey + ScoreConfigMaintWorkMem in Config.cs; the stamp in PgTargetFactCollector.Vacuum.cs (facts.Find(ConfigMaintWorkMem)?.Metadata[…] = ratio, 3 lines beside facts.Add(fact)) 0 unstamped; 0 at ratio 0.5; 0.4 at 1.0; no lineage stamp (engine-defined); not an advisory root; ScoreAll: 0.6 beside the fired backlog, 0.4 beside a non-persistent one, 0 unstamped beside a fired one; source pin on the stamp
11 Parked offered-vs-delivered hook reads Metadata["trend"] (3, addendum) Rename inside the comment to TpsTrendKey; stays parked PgTargetScorer.Sessions.cs comment Source pin: names the constant, no "trend" literal

Decided NO-OP (from the list): wal_level / archive_mode — not in §4c, later slice. Per-table CONFIG_PG_AUTOVACUUM_DISABLED key — v1 folds the reloption into the backlog fact (metadata + amplifier + advice), no key. The tps_trend co-fire uncomment — waits on the sessions family wiring and pinning the co-fire against lane 6's PG_TPS trend; item 11 only makes the parked text name the shipped constant.

Why this shape

  • GetActiveEdges virtual, not hidden and not a collector-stamped alias fact. InferenceEngine holds a RelationshipGraph reference and calls GetActiveEdges on it (Traverse, and the union-find in GroupIncidents), so a new method on the subclass would never dispatch; and the alternative lane 7 recorded — the collector stamping a real PG_BAD_ACTOR fact — puts the alias INTO story paths and root_fact_key, duplicates a fact, and loses the drill-down (keyed on the concrete key). One word on the base (the same kind of seam PostgreSQL targets enter the analysis pipeline: the pass routes by registry engine_kind inside the service, a pg_database_stats coverage witness replaces the tombstone, and every shared switch gains its one pg_ arm so the content lanes can build in parallel (#3542 v1 plumbing) #3665 opened with AddEdge private → protected and the (bool) ctor), the SQL Server graph's class does not override, the base body is untouched, and PgTargetSharedSwitchRoutingTests' seam pins (ctor / AddEdge accessibility) still hold. Pinned: the base method IsVirtual, the override's DeclaringType, and that RelationshipGraph's own declaration is the base's.
  • Severity, not base severity, for the resolution — the traversal orders candidates by Fact.Severity, and the alias should land where the walk would have gone had every statement been nameable.
  • maintenance_work_mem's evidence is the backlog RATIO, gated at 1.0. The work_mem twin grades a spill RATE against an unmeasured floor and stamps threshold_lineage = 0; this one grades the backlog against the table's own autovacuum_vacuum_threshold + scale_factor × reltuples line — the number PostgreSQL itself acts on, and the same engine-defined line the backlog fact's concerning bar sits on — so it is engine-defined and carries no unmeasured stamp (the class summary's rule for this file). The stamp lives in the vacuum collector because ScoreConfigFact(Fact) receives one fact and no lookup: the evidence has to be ON the fact, and only the collector that computes the backlog can put it there. This is a three-line edit to lane 4's (merged) collector, mirroring Database.cs's stamp line for line; flagged below.
  • The saturation advice edit is the falsified-line rule, not scope creep. The ceiling now subtracts a number the sentence "max_connections 100 minus superuser_reserved_connections 3 — that is 90 / (100 − 3) = 93%" did not mention; on a 16+ target that set reserved_connections, the displayed arithmetic would disagree with the ratio beside it. The conditional adds the term only when non-zero; every existing pin on the sentence is byte-for-byte.
  • autovacuum = off at 0.9 is a posture band, argued in the lineage comment: the setting is the evidence (as fsync = off is for pg_posture), above the incident line on its own, below 1.0 because the backlog co-fire is what says the damage is measured rather than pending.

Censuses that fired, and how each was satisfied

  • PgTargetThresholdLineageTests — both new comparison lines use only trivial literals (0 / 0.0 / 1.0) and still carry a marker within six lines (engine-defined / posture on the 0.9 const; engine-defined on the 1.0 trigger line). Green.
  • PgTargetFactCollectorTests.AllSql_IsEveryPublicSqlConst_… — item 6 edits PgTargetConfigSnapshotSql, a public const already in the reflection closure. Green. The FROM/JOIN census is item 4's edit. The 12-name collect-surface census, one-file-per-family and filled by lane N — untouched, green.
  • PgTargetMcpSurfaceTests.EveryPgRecommendation_NamesARegisteredTool_… — fired twice: ConfigReservedConnections (a real key) needed a row → added (ConfigReads()); BadActorFamily (an alias, never a story key) → the census now excludes it through IsBadActorAlias, with the reason inline. Edited deliberately.
  • PgTargetSharedSwitchRoutingTests — delegation equality unchanged (the alias delegates null == null; no static block added). Green, no edit.
  • PgTargetKnobsTests — its "other lanes' keys fall to 0 … until they fill it" line pinned ConfigAutovacuumOff = 1 → 0; now filled, so the pin says so and grades the unstamped maintenance_work_mem, autovacuum = on and reserved_connections at 0 instead; the snapshot name list pin gains reserved_connections. Edited deliberately.
  • PgTargetQueriesTests.TheNextReads_ForABadActor_… — the two-tool order → three, duration trend first. Edited deliberately.
  • StoreSqlClockDisciplineTests, RepoFileAdoptionTests (the new test's anchors are single-line, so no s_lfReaders entry), FactSourceRegistryTests, CommentFilterAdoptionTests, FactCollectorCommandTimeoutTests, FactCollectorFailureReportingTests, PgTargetPostureIsolationTests — green, no edit.

Blast radius

SQL Server pass: RelationshipGraph.GetActiveEdges gains the virtual modifier and nothing else; no SQL Server class overrides it; InferenceEngine is untouched; the seven Lite tests constructing RelationshipGraph() compile and the Lite.Tests build is green. ComparisonBanding is a comment. PostgreSQL pass: a target with autovacuum = off now gets a 0.9 posture card it never got; a target whose backlog fired now sees CONFIG_PG_MAINT_WORK_MEM at 0.6 in the backlog story's path (previously 0, invisible); a 16+ target with reserved_connections > 0 gets a lower (correct) ceiling and one more sentence term; every other target's ceiling and advice are byte-identical (the knobs live e2e plants no reserved_connections row — its 13-fact pin stands; the vacuum live e2e plants no config snapshot — no stamp, its path pin stands). No schema, no rung, no StorageVersion bump.

What it does NOT do

No edge INTO the alias is written here — lanes 6 and 9 own those (AddEdge(PG_TEMP_SPILL, PgTargetFactKeys.BadActorFamily, …) is the whole change on their side; the test arranges exactly that edge through the protected AddEdge). No PG_MONITORING_PERMISSIONS change (item 5). No numbackends (a rung; item 9 goes on the issue). No wal_level / archive_mode, no per-table autovacuum key, no tps_trend uncomment. No Lite twin (D1). Lane 9 is in PgTargetScorer.Anomaly.cs, PgTargetAnomalyDetector, PgTargetBaselineProvider, AnomalyThresholds, MetricNames concurrently — none touched here; my PgTargetFactKeys.cs additions are one delimited block after lane 7's prefix, so whichever of us merges second rebases around one hunk.

Boundary extensions (load-bearing, outside the named files — each one line)

  1. PerformanceMonitor.Analysis/RelationshipGraph.cs — GetActiveEdges → virtual (+ a three-line doc note). Required: the override is the only place a destination can be rewritten before InferenceEngine reads it.
  2. Darling/PerformanceMonitor.Darling.Analysis/PgTargetFactCollector.Vacuum.cs — the backlog-ratio stamp onto the knob fact (3 lines). Required: amplifiers cannot lift a base of 0, and only this collector has the ratio.
  3. PerformanceMonitor.Analysis/PgTargetAdvice.Sessions.cs — the conditional third term in the arithmetic sentence. The falsified-line rule.
  4. PerformanceMonitor.Analysis/PgTargetRelationshipGraph.Query.cs — one sentence recording that lane 7's open decision is now made.
  5. Tests: PgTargetKnobsTests, PgTargetQueriesTests, PgTargetMcpSurfaceTests — the three pins the items necessarily move, each with its reason inline.

Test plan

New Darling/Darling.Tests/PgTargetBetweenWavesTests.cs (14 tests): the alias is never a key and no arm claims it; ResolveBadActor higher-severity / ordinal tie / null; the alias edge through the real InferenceEngine.BuildStories (path PG_TEMP_SPILL → PG_BAD_ACTOR_7, stored edge keeps the alias, a second call resolves afresh) and dropped-not-thrown with no bad actor; non-alias edges pass through as the same instances + the virtual/override/declaring-type seam pins; compare_analysis presence path; autovacuum = off 0.9 / 0 / no lineage stamp / advisory root, ScoreAll alone 0.9 and 1.35 beside the fired backlog; maintenance_work_mem 0 / 0 / 0.4 by stamp, no lineage stamp, not a root, ScoreAll 0.6 / 0.4 / 0 in the three arrangements, and the collector stamp by source; reserved_connections in the SQL, its key, base 0, the ceiling line and mandatory-pair gate by source, pg_use_reserved_connections named; the saturation advice byte-identical at 0 and honest at 2; the bad actor's next-reads lead with a registered get_pg_query_duration_trend the advice also names; the parked hook names TpsTrendKey.

Executed here, not only compiled — a net10.0 harness (AssemblyName=Darling.Tests) compiling the new class plus PgTargetThresholdLineageTests, PgTargetFactCollectorTests, PgTargetSharedSwitchRoutingTests, StoreSqlClockDisciplineTests, RepoFileAdoptionTests, PgTargetKnobsTests, PgTargetQueriesTests, PgTargetVacuumTests, PgTargetTempTests, PgTargetSessionsTests, PgTargetMcpSurfaceTests, PgTargetPostureTests, PgTargetPostureIsolationTests, FactSourceRegistryTests, CommentFilterAdoptionTests, FactCollectorCommandTimeoutTests, FactCollectorFailureReportingTests, AnalysisPassCommandTimeoutTests, AnalysisPassTokenThreadingTests, LiveCleanupConversionRatchetTests: 311 / 312 green, the one red being the harness's own output-directory walk-up in LiveCleanupConversionRatchetTests (cannot locate Darling/Darling.Tests from the harness binary — an artifact, the same class of red lane 7 reported for DocCommentHygieneTests); the six DARLING_TEST_PG-gated live e2es were skipped here (no rig raised — item 6 adds a name to an IN list and the reserved_connections setting is a documented PostgreSQL 16 GUC; CI's Darling PostgreSQL tests job executes those e2es, and the sessions one is the live proof that an ABSENT reserved_connections row still yields 97 usable).

Builds, zero warnings of ours: PerformanceMonitor.Analysis, PerformanceMonitor.Darling.Analysis, PerformanceMonitor.Darling.Service, Darling.Tests, Lite.Tests (-c Release -p:EnableWindowsTargeting=true). The one Lite.Tests warning (LogTailOverlapThresholdPinTests.cs:95 xUnit2000) is pre-existing on dev.

Which component(s) does this affect?

  • Lite
  • Darling
  • Lite Tests
  • Darling Tests
  • SQL collection scripts
  • Documentation
  • Full Dashboard (deprecated)
  • CLI Installer (deprecated)

How was this tested?

Above. No production store or fleet server was read. No live rig was raised for this PR; the gated e2es run in CI.

Checklist

  • I have read the contributing guide
  • My code builds with zero warnings (dotnet build -c Debug)
  • I have tested my changes against at least one SQL Server version (not applicable — the one SQL-Server-reachable edit is a virtual modifier on a method no SQL Server class overrides; the SQL Server graph's behaviour is the unchanged base body, and the Lite tests that construct it build green)
  • I have not introduced any hardcoded credentials or server names

Partial for #3542 (between waves; the shared-file batch). CHANGELOG deliberately untouched — the coordinator splices this wave's entries.

…orted back: a stable PG_BAD_ACTOR alias the graph resolves to the window's heaviest statement at story-build time, the two vacuum-side config facts that were inert because no arm ever gave them a base (autovacuum = off is a 0.9 posture fact; maintenance_work_mem is evidence-gated the way work_mem is), PostgreSQL 16's reserved_connections in the connection ceiling, and the bad actor's first next-read (#3542, between waves)
@erikdarlingdata
erikdarlingdata merged commit 340998e into dev Sep 19, 2026
6 of 7 checks passed
@erikdarlingdata
erikdarlingdata deleted the feat/3542-between-waves branch September 19, 2026 03:28
erikdarlingdata added a commit that referenced this pull request Sep 19, 2026
…le, Opus full tier (#3710) (#3718)

* REVIEW-TRAPS.md: the repository's recurring defect shapes, one line each with the PR that bit (#3710)

Seven layers (SQL, collectors and deltas, analysis, alerting and notifications,
storage and MCP payloads, tests and censuses, workflows and CI, repository
mechanics), fifty-odd lines, every one citing the pull request or issue that
verified it: the >= limit truncation inference (#3594/#3679/#3699), ORDER BY
3 + 4 folding to a constant (#3613), MAX(max_dop) across plans (#3648), integer
division in cntr_value_per_second (#3653 Q9), per-collection alert firing
against a shorter cooldown (#3628/#3636), restart zeros in deltas and baselines
(#3595/#3698/#3705/#3708), ScoreAll skipping amplifiers on a zero base and the
non-virtual GetActiveEdges (#3688), the PgTargetFactKeys censuses (#3688),
pg_total_relation_size vs the heap and the two TimescaleDB views that hide
materializations and history (#3610/#3585), the -infinity crash backoff (#3629),
the Slack block budget and the surrogate-pair cut (#3612/#3618/#3625/#3644),
LCK_M_SCH_M folding into LCK so no SCH_M fact exists (#3709), one wait profile
per run (#3709), and the CRLF round-trip. Each citation was read before it was
written down; a wrong citation is worse than none.

The review workflow reads this file before the PR body and must say, per trap
the diff is near, whether it applies here and why.

* Claude review: blind read before the body, callers rule, structured verdict with a ledger, and an Opus/Sonnet tier by changed path (#3710)

The prompt runs in two phases. Phase 1 forbids gh pr view / git log / git show
until the reviewer has read the diff, every changed region in context, the
callers of every changed public symbol (Grep, both SKUs and both test projects)
and .github/REVIEW-TRAPS.md, and written its own account. Phase 2 reads the
body and grades every claim VERIFIED / UNVERIFIED / REFUTED, then reports the
DIVERGENCE between the two accounts. The verdict is a fixed block -- Claims,
Riskiest lines with the pinning test or UNPINNED, Traps near this diff,
Divergence, Reviewer checklist -- ending in one machine-readable ledger line,
posted through gh pr review --body-file - with a quoted heredoc (no Write tool;
the allowlist is unchanged and still read-only).

The #3650 verdict check now fetches the newest verdict in the window by id and
fails the job when the body lacks the "## Verdict:" header or the ledger, or
when the ledger's verdict disagrees with the review's API state (a
"CHANGES REQUESTED" body submitted with --comment is the shape the guard would
wave through). The guard's own arm is untouched and still keys on the state.

A dorny/paths-filter@v4 step (the repo's pin style) routes changes under the
shared libraries, Darling Storage/Analysis/Service, Lite Services/Mcp/Database,
install/ and every .sql file to the full tier -- claude-opus-5[1m], 60 turns --
and everything else to claude-sonnet-5 at 30 turns; one decision step emits
tier/model/max_turns, one review step consumes them (two steps would double the
"review posted" accounting the guard does), and the prompt carries the tier and
model into the ledger. A filter that dies degrades to FULL, not light.

Step name "Claude review" and the ALWAYS POST sentinel are unchanged: both are
read by claude-review-guard.yml.

* Review tier: Alerting and Notifications read at the full tier (#3710)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant