Skip to content

Add the remediation credential seam, the journal's actor and the evict-then-observe state machine (#2138 phase 1) - #3170

Merged
erikdarlingdata merged 14 commits into
devfrom
feature/2138-phase1-remediation
Sep 8, 2026
Merged

erikdarlingdata merged 14 commits into
devfrom
feature/2138-phase1-remediation

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Sep 8, 2026

Copy link
Copy Markdown
Owner

The credential seam, the journal's actor, and the shared evict-then-observe state machine for #2138 phase 1. This branch cannot write to a monitored serverPlanForceNoWritePathTests stays green, verified by scanning the built service assembly with a working positive control, not by reading the diff.

What #2138's design asked for, and what dev already had

The design pass (#2138, 2026-08-18) specifies phase 1 as four pieces: the remediation credential seam, a remediation_actions journal in both stores, the evict/observe/force flow, and a viewer surface. Two of its premises have moved since it was written.

The journal already exists. #2745 (merged 2026-09-01) built it as collect.plan_force_actions at V107, with the design's exact column list — server, database, query/plan keys, evidence snapshot, outcome — and its doc comment calls it "the bot's audit trail and ledger". Adding a second remediation_actions table beside it would be the table-shaped version of the second policy path this design exists to avoid, so this branch extends that one instead. The design's own words are "built once, in Phase 1"; it was, under a different name.

The bot already consumes it in dry run. #2745 also landed ForcePlanBotPolicy, ForcePlanSelfReview, the cooldowns and the would-force ledger — design phase 2's brain — with IPlanForceExecutor declared and unimplemented. #2731, which would have armed it, was closed on 2026-09-07 with the ruling that the write path is "genuinely outstanding and deliberately fenced" and phase 2 "should be rewritten against dev's seam".

The credential seam

V113 adds remediation_username / remediation_encrypted_password to config.config_monitored_servers, beside username / encrypted_password: same DPAPI-LocalMachine blob, same env:/file: reference support, same --encrypt-password. Both nullable, no default, no fallback to the monitoring credential — that credential stays read-only forever, which is a promise both READMEs and the MCP instructions make and operators grant against.

Presence of the username is the auth mode. There is deliberately no remediationAuth sibling: an integrated remediation identity would be the service account, which is the monitoring identity, which is the thing this exists to keep read-only. And no plaintext slot: Password's dev-convenience arm exists because a wrong monitoring password fails a read, and a wrong remediation password fails a write against production.

ResolveRemediationPassword returns null for an unarmed server rather than throwing, unlike its monitoring twin. A missing monitoring password is a misconfiguration; a missing remediation password is the shipped state of every server, and an exception in the normal case is one callers learn to swallow.

BuildRemediationConnectionString is a separate function rather than a flag on the existing builder — a useRemediationCredential: true parameter would put the write identity one mistyped argument away from every collector. It presents ApplicationName = "PerformanceMonitorDarling-Remediation", which is the audit trail on the server's side: a DBA reading sys.dm_exec_sessions during an incident has to be able to tell the one connection that can change a plan from the forty that cannot.

How a server without one has no surface, rather than a disabled one

OperatorRemediationGate.SurfaceFor returns OperatorRemediationSurface? and returns null. The nullability is the mechanism, not a convention: a record carrying Enabled = false is one IsEnabled binding away from a greyed-out button with a tooltip, whereas a view-model holding null has nothing to bind a command to and nothing to draw. TheSurfaceIsExpressedAsANullableReturn_NotAnEnabledFlag reads the return's nullability through NullabilityInfoContext and rejects any Enabled/Disabled/Visible member on the surface type, so the shape that makes a disabled control easy to write fails in CI.

It executes the existing verdict object

The gate takes a StructuredForcePlanTarget and reads Eligible and Blockers — the fields FactRemediation.BuildStructuredRemediation fills from FactRemediation.ForcePlanBlockers, the same output an MCP consumer reads and the same function PlanForceBot already passes through. Nothing in the gate looks at ParameterSensitivityCoFired, ReplicaRole, or any other evidence field.

It reads both halves, and disagreement refuses. On any object the projection produced they agree (Eligible is defined as blockers.Count == 0) and the second read is free; on a hand-built or deserialized target it is not, and reading one alone would arm a blocked target on a flag. AVerdictWhoseHalvesDisagree_GetsNoSurface pins both directions.

The PSP never-auto-force contract arrives through the projection rather than being restated: AParameterSensitiveTarget_GetsNoSurface_EvenFullyArmed builds a real ForcePlanTarget, projects it, asserts the verdict really carries parameter_sensitivity_cofired (so the case cannot pass for the wrong reason), and then asserts no surface.

The journal's actor, and the own-forces-only invariant

V113 adds collect.plan_force_actions.actor. GetPendingReviewsAsync' own-forces-only property was documented as structural because "the read starts from rows this bot journaled" — true only while the bot was the table's only writer. An operator writing to the same table makes the bot's self-review able to find an operator's force, judge it against evidence it never saw, and take it back, breaking the standing house rule in the direction nobody notices until a hand-pinned plan quietly stops being pinned. The read is now AND pfa.actor = 'bot'.

This is the shape phase 2's own-forces-only rule needs. Phase 2's unforce cannot reach a row it cannot join to a bot-actored force row: the review read filters on the actor, and the terminal-row check joins on related_action_id, which is the table's own identity PK. So "only forces this bot placed" is two predicates on one table rather than a rule someone has to remember — the foreign-key-shaped invariant the design asked for, available before the bot can place anything.

The DEFAULT is added and then dropped. Every existing row was written by the bot, so DEFAULT 'bot' backfills them correctly; leaving it would make an INSERT that forgets actor silently claim to be the bot, which is the actor whose forces the review may unforce. Dropping it fails that INSERT loudly. PlanForceActionRecord.Actor is a required member for the same reason — the compiler enumerated both construction sites via CS7036 rather than a reviewer having to.

GetQueryHistoryAsync is deliberately not actor-filtered, and the asymmetry is documented so it does not get "fixed" for symmetry. That read restrains the bot, and every limb restrains it correctly by counting an operator's rows: a query a human touched two hours ago is a query the bot should stay off, an operator's force spends real blast radius, and an operator's force that would not stick is evidence the next one will not either. Filtering would make the bot more willing to act the more a human already had.

The flow, and both click boundaries

OperatorRemediationFlow.Observe is pure and static, no clock and no I/O, matching ForcePlanBotPolicy. Nothing in it executes anything: the eviction happened before the caller could ask, and ForceOffered is permission to draw a control, never permission to act.

  • First boundary — the operator arms the eviction. Journal intent, evict, observe.
  • Observation window: whichever comes first of MinReviewExecutions executions or ObservationWindowMinutes. The executions floor is not restated — Observe takes it as a parameter and callers pass ForcePlanBotSettings.MinReviewExecutions, the one named home for detection's 25. ObservationWindowMinutes = 30 is new; there was no 30-minute detection floor to reuse, and it lands on the bot's settings so a human and the bot observe one eviction for one length of time.
  • Second boundaryForceOffered is true for exactly one verdict, RegressedPlanReturned, and it is a property of the returned value so no caller can re-derive it differently. ExactlyOneVerdictOffersTheForce checks it over every verdict by reproducing each one through the machine and reading ForceOffered back, rather than against a table retyped in the test.

The two limbs are not interchangeable, and that is the design's substance. Plan identity is not a statistical quantity — one compile settles which plan the optimizer chose — so the regressed plan returning is a legitimate verdict on a window the timeout closed with three executions, and it is the arm that justifies the force. Cost is statistical and gets the executions floor. A window the timeout closed below the floor is observation_inconclusive, journaled as its own outcome: folding it into RegressedPlanReturned would offer a force on the strength of not having looked. Every outcome journals (optimizer_recovered, regressed_plan_returned, worse_after_evict, observation_inconclusive).

Plan-handle FREEPROCCACHE per platform, and the named degrade

The honest answer is that a per-platform table is the wrong instrument, so this branch does not ship one. Two facts have to hold and they fail for reasons an operator would fix differently:

  • The engine has the statement. DBCC FREEPROCCACHE does not exist on Azure SQL Database (EngineEdition 5); Managed Instance has it. No grant changes that.
  • The credential holds ALTER SERVER STATE. A grant does change that.

Collapsing them would tell an operator to ask a managed provider for a permission that would not help. EvictCapability keeps them separate and EvictUnavailableReason resolves platform before permission, pinned by PlatformOutranksPermission_SoNoOneIsSentToAskForAGrantThatCannotHelp.

The permission half is probed, not predicted: AlterServerStateProbeSql is SELECT has_alter_server_state = has_perms_by_name(NULL, NULL, 'ALTER SERVER STATE');, run read-only as the remediation credential. That answers for the effective permissions of the executing login, which is the only question that matters — the grant can arrive through a server role, through CONTROL SERVER, or directly, and an operator on a managed platform generally cannot tell which they were given. A table of "platforms that allow plan-handle FREEPROCCACHE" is a claim about a managed service's grant set, which the vendor can change without telling us, and it would go stale in the direction that keeps passing.

I did not execute a DBCC FREEPROCCACHE against any server to settle this — that is a write to production and outside what this branch does at all. So the permission question is answered by the shipped probe rather than by an assertion in this description. TheCapabilityProbeIsReadOnly pins that the probe is a single SELECT with no DDL keyword, asks the server-scope form (both leading arguments NULL, or it answers about the current database and reports a grant an eviction cannot use), and aliases its output column.

The degrade is pinned as never-silent, exhaustively. NoCapabilityShapeCanDegradeWithoutANamedReason walks all six EvictCapability shapes and asserts that a surface not offering evict-first always carries a reason and one offering it never carries a stale one, with a count check so a comprehension that produced nothing cannot pass by asserting about no cases. ExactlyOneCapabilityShapeOffersEvictFirst states the grant as a count, so quietly widening it — treating an unprobed capability as permitted, say — fails here rather than surfacing as an unexpected eviction attempt. An unprobed capability is its own reason (evict_capability_unknown), deliberately distinct from denied: refusing to guess rather than attempting a write to find out.

Evidence

The pins run. Darling.Tests and Lite.Tests target net10.0-windows and cannot execute on macOS, so the actual test sources — not copies — were compiled into a net10.0 console harness over a minimal xUnit shim and executed against the real assemblies: 93/93 passing, covering the two new suites plus ForcePlanBotPolicyTests, ForcePlanSelfReviewTests, MigrationDataMovingRungCensusPins and MigrationLadderPins.

Red-then-green on the census pin. The V113 CREATE INDEX on the populated collect.plan_force_actions is a new data-moving rung; the real pin failed with exactly that finding before it was declared, and passes after. It also measured that the rung's ADD COLUMN ... DEFAULT 'bot' and ALTER COLUMN ... DROP DEFAULT are not findings, rather than my reasoning that they would not be. Declared SetsTheFloor: false with the arithmetic: the table's size is capped by the bot's cooldowns and a 365-day horizon at roughly 46k rows across a 42-server fleet, and it is empty on every store today.

Mutation: 11/11 killed, each with a before→after content hash proving it applied and each naming the test that caught it.

Mutation Killed by
credential check removed AServerWithNoRemediationCredential_GetsNoSurface
Blockers half of the verdict dropped AVerdictWhoseHalvesDisagree_GetsNoSurface(true, …)
Eligible half of the verdict dropped AVerdictWhoseHalvesDisagree_GetsNoSurface(false, …)
degrade precedence inverted PlatformOutranksPermission_…
unprobed capability treated as permitted AnUnprobedCapability_…, ExactlyOneCapabilityShapeOffersEvictFirst
limb order swapped (timeout beats evidence) WhenBothLimbsAreSatisfied_TheOneCarryingEvidenceWins
plan identity gated behind the cost floor TheRegressedPlanComingBack_IsJudgedOnIdentity_EvenOnTheTimeoutLimb
force offered on optimizer recovery too ACheaperDifferentPlan_…, ExactlyOneVerdictOffersTheForce
exec floor hardcoded to 25 TheExecutionsLimbFiresAtTheSettingsFloor_NotAtALiteral
empty plan hashes match each other AnAbsentRegressedHashOnTheTargetSide_AlsoNeverMatches
dead band removed ADifferentPlanIndistinguishableInCost_IsInconclusive

The first pass reported 7 of the 11 as NOT-APPLIED rather than as kills, because the new source files are LF in the working copy while the anchors were CRLF. That is the failure mode worth naming: a mutation that never applied is indistinguishable from one that was caught, so the harness asserts the anchor count and the content hash before crediting anything.

Two of my own tests were wrong and the harness found them. TheCapabilityProbeIsReadOnly failed twice on the real statement for reasons that were the scan's fault: first the permission name 'ALTER SERVER STATE' matched as a DDL keyword, then the output alias has_alter_server_state did, because a substring scan cannot tell an identifier from a statement. It now strips quoted literals and matches on word boundaries, with a positive control that it still fires on a real write. A scan with either flaw has to be silenced to ship, and a silenced scan guards nothing.

The fence is green, measured. The built PerformanceMonitor.Darling.Service.dll was decoded at both UTF-16 alignments and searched for all three write statements: none present. The scan's positive control found the new PerformanceMonitorDarling-Remediation literal in the same assembly, so it does read real literals.

Not verified locally: the live-Postgres own-forces-only scenario in PlanForceActionStoreTests (it needs the Darling PostgreSQL tests job), and the Lite.Tests divergence pin's schema half (Schema.GetAllTableStatements() is in the net10.0-windows Lite project). Its two source-scan halves were replicated in Python: 266 Lite .cs files, zero hits on any of the 13 capability tokens.

CI

All eight checks green on 162b615d0. Durations are the API's started_at/completed_at, not wall-clock guesses.

Check Duration What it means here
check-branches 3s base is dev
description-drift 8s
Darling whole-tree guards 16s
Darling Linux build 1m50s it compiles
Darling PostgreSQL tests 3m37s V113 applied against a live store; the live own-forces-only scenario ran
review 6m08s claude[bot]; findings below
verify 6m42s
build 8m54s Windows: Lite.Tests 3494 / 0 failed / 0 not run, Darling.Tests 8122 / 0 failed / 0 not run, Dashboard.Tests 782 / 0

build at ~9 minutes is a real run, not the docs fast path, and its step list confirms it: Run Lite tests, Run Dashboard tests and Run Darling tests all executed rather than being skipped.

The new tests ran, counted two ways. Darling.Tests went 8115 → 8122 across the commit that added RemediationCredentialRungTests, which is exactly its seven [Fact]s; and none of the new suites appears in the SKIP list. MTP prints no per-test pass lines, so Failed: 0 + Not Run: 0 + absent-from-SKIP is the available evidence.

What CI caught that local verification could not

Five failures on the first run, all in the class the Windows-only and live-Postgres suites exist to find. Recorded because four of them are the registration ratchets any new rung trips, and the fifth is the rung working.

  1. DarlingRetentionTests end-to-end: 23502 null value in column "actor". Its raw INSERT into plan_force_actions omitted the column. That is the dropped DEFAULT doing exactly what it is for — the INSERT that forgets the actor fails instead of silently claiming to be the bot. The test now names it.
  2. CollectorStallProbeStoreTests and CollectorStallProbeViewerGateTests asserted V112 was the top rung, which V113 makes false. That claim belongs to whichever rung actually is top, or every new rung breaks every older rung's suite, so it moved to RemediationCredentialRungTests: the all-sentinels-true mapping, the last-ordinal equality, and the one-rung-behind check. V112 keeps its own fixed ordinal and now switches off every LATER sentinel for its behind-check, which generalises to the next rung rather than needing this edit again.
  3. DarlingManagedRolesTests pins the exact secret-column list; remediation_encrypted_password had to be argued onto the secret side rather than waved through.
  4. RegisteredServerSettingDriftTests pins the drift-exclusion list literally, so growing it is a visible act — a good guard, and its own point is that a bumped literal should not be enough. The two new keys are excluded because a drift report is what triggers a disconnect-and-reconnect of the MONITORING connection, and tearing down collection because someone rotated a credential collection never uses would be a real outage. Since bumping a literal by reflex is how such a guard stops meaning anything, the property behind it is now asserted too: only a credential-shaped key may be excluded, so excluding trustServerCertificate — the field Editing a registered server's settings in darling.json is silently ignored — only ADDING a server is warned about #2552 actually reported — fails rather than passing with a bigger number.
  5. The Lite divergence pin's schema half was over-broad for the second time in one file. plan_correction carries a generated last_good_plan_force_failure_reason column, so a statement-substring scan reported the monitored server's own Query Store forcing-failure reason as a bot journal. It now parses table NAMES, with plan_correction as the control proving the parse discriminates a column from a table.

Review finding, addressed. claude[bot] caught that GetPendingReviewsAsync' XML summary still said own-forces-only was "structural — the read starts from rows this bot journaled". True while the bot was the table's only writer, false as of V113, and directly contradicting the SQL comment a few lines below it. A stale WHY-comment here is worse than none: it invites removing the actor filter for looking redundant, which is the guarantee it now carries alone. Fixed in 162b615d0. The final review pass reports no blocking issues.

Lite: Darling-only, permanently — and the divergence is asserted, not inherited

Settled on #2138: remediation execution is Darling-only, permanently. The design's "one PR-sized unit per SKU pair" does not apply, and this is a wording fix rather than a reversal.

The rule was already stated in Lite's own code, but with two different reasons, only one of which survived — which is the case that shows why this pin asserts rule and reason separately. The obsolete half named the deprecated Dashboard SKU as the owner of remediation execution, so it pointed at nothing live. The half that decides it is Lite's store: a DuckDB file is per-workstation, so a remediation journal kept there cannot be the shared audit trail such a journal exists to be — two operators acting on one server would each hold half of it and neither could see the other's forces — and it cannot carry phase 2's own-forces-only invariant, which is a predicate plus a related_action_id self-reference within one shared store. No deprecation can invalidate that, so all three sites that state the rule now rest on it.

The Dashboard mentions in LiteRecommendationsViewModel and RecommendationsTab stay: they are UI comparisons about an "Open in Active Queries" deep-link, not scope claims, and their reason is the surface's tab shape. They now also serve as the positive control proving the dead-reason absence assertions are not passing against a failed read.

OperatorRemediationLiteDivergencePinTests asserts the divergence is complete and its stated reason still exists, failing in opposite directions: Lite gaining a credential, executor seam or journal fails the capability scan, and the advise-only statements being deleted or reworded away fails the last test, so this pin cannot outlive its own premise. It also asserts the shared seam is reachable from Lite and lives in the same assembly as FactRemediation, so whichever way the call goes no logic is written twice.

A Lite journal table was deliberately not added. A plan_force_actions in DuckDB while Lite cannot act would be a permanently-empty table, which reads to every later consumer as "the feature is here and nothing has happened" rather than "the feature is not here" — the exact one-SKU-empty-value trap. V107's doc comment already keeps its column shape twin-ready.

What is not in this branch

  • IPlanForceExecutor's implementation — the force/unforce/evict statements, and the orchestration that drives Observe from live Query Store deltas. This is the change that necessarily opens PlanForceNoWritePathTests, and per that file's own remarks the fence should be relaxed in the diff whose subject is making its claim stop being true. Opening it for a half-flow, while the SKU question above is unsettled, is the wrong order.
  • The viewer surface — the command-plane verbs and the card affordance. DarlingCommandExecutor's switch is where a new operator-armed verb lands; RecommendationCardViewModel already carries the RemediationAction a surface would gate on.
  • execute_remediation — explicitly out of scope per the design's sequencing.
  • The plan_force_bot_enabled viewer surface V107 promised ("viewer surface to follow"), still absent.

Open, and yours

  • ⚖ MCP write verbs at all — the only one of the three still open. Nothing in this branch presumes either way.

Settled since this PR opened: Lite stays advise-only permanently, and the ship vehicle is whenever the next release is cut (it only bites once the write path exists). Both recorded on #2138.

CHANGELOG entry text (uncommitted)

- **Remediation credential seam and journal actor (#2138 phase 1)**: monitored servers can carry a
  second, per-server, opt-in remediation credential (`remediation_username` /
  `remediation_encrypted_password`, V113) so an operator-initiated action never travels on the
  read-only monitoring credential. A server without one has no remediation surface at all rather than
  a disabled one. `collect.plan_force_actions` gains `actor`, and the force-plan bot's pending-review
  read now filters on it — the bot's self-review can only ever take back its own forces, not an
  operator's. Adds the shared evict-then-observe state machine (observation window: whichever comes
  first of 25 executions or 30 minutes) and the named-reason degrade when targeted plan-cache
  eviction is unavailable, either because the engine edition lacks `DBCC FREEPROCCACHE` or because
  the remediation credential lacks `ALTER SERVER STATE`. No write path to a monitored server ships in
  this change.

…ase 1)

The monitoring credential stays read-only, so an action travels on a second,
per-server, opt-in remediation credential or it does not travel. V113 adds
remediation_username / remediation_encrypted_password to the registry, both
nullable with no default and no fallback: a null credential is the shipped state
of every server and means that server has no phase-1 surface at all.

OperatorRemediationGate.SurfaceFor returns a NULLABLE surface rather than one
carrying an enabled flag, so the absence is unrenderable instead of being a
disabled control that explains itself. It reads Eligible AND Blockers off the
existing structured_remediation verdict object and asks no evidence question of
its own.

V113 also adds collect.plan_force_actions.actor. GetPendingReviewsAsync' own-
forces-only property was structural only while the bot was the table's only
writer; an operator writing to it makes the bot's self-review able to find and
unforce an operator's force. The read is now filtered on actor = 'bot', and the
record member is required so the compiler enumerates construction sites.
…nvariant

The gate's pins go at the two properties that are easy to lose: absence is
expressed as a nullable return (read through NullabilityInfoContext, so a change
to a non-nullable return carrying an enabled flag fails), and no capability shape
can withhold evict-first without a named reason (exhaustive over all six).

The flow's pins separate the two window limbs by the evidence each can support:
the regressed plan returning is judged on identity and so is valid on the timeout
limb, while a cost verdict is not. The executions floor is driven off
ForcePlanBotSettings rather than a literal, and raising the setting has to move
the boundary — so a restated 25 fails.

The own-forces-only test journals an operator force shaped exactly like the bot
force the read returns, alongside a bot-actored twin: without the twin, a filter
matching nothing would satisfy the first assertion perfectly.

The Lite divergence is asserted rather than inherited. Its first form scanned for
"remediation" and found 12 correct hits, because rendering advice is a Lite
feature; it now names the capability tokens instead, and checks the advise-only
scope statements it rests on still exist.
V113 accepts a per-server remediation credential and nothing in this build
consumes it, so an operator who entered one would believe the server was armed.
A knob that silently does nothing is at its worst when what it claims to gate is
a write to a production server, so the credential is accepted, stored,
resolvable, and announced as inert at connect — the same discipline #2745 applied
by journaling an all-gates-open force as WITHHELD rather than downgrading it.

A one-sided credential gets its own warning: "you configured half of one" and
"this build cannot use it yet" send an operator to different places.

darling.sample.json documents both keys with the grants each lever needs, that
ALTER SERVER STATE is genuinely optional, and that the service probes for it
rather than guessing per platform.
@claude

claude Bot commented Sep 8, 2026

Copy link
Copy Markdown

Review summary

Reviewed the diff against dev (22 files, ~1896 additions). This is Darling-only work (credential seam, journal actor, evict-then-observe state machine); the Lite divergence pin (Lite.Tests/OperatorRemediationLiteDivergencePinTests.cs) correctly asserts Lite stays untouched rather than silently drifting out of parity, so I didn't flag a parity gap here — the "no Lite change" is itself pinned and tested.

Checked in detail: the V113 migration (PgMigrations.cs / StorageVersion.cs, schema-qualified, DEFAULT-then-DROP pattern, matching index), the four-part migration-rung checklist from CONTRIBUTING.md (rung, version, pins, viewer probe — all present and consistent), the credential seam (DarlingSecrets.ResolveRemediationPassword, MonitoredServerConnection.BuildRemediationConnectionString, DarlingConfig.MonitoredServer), the StoreConfigProvider seed/read SQL (parameter-to-column mapping for the 23-column INSERT and the read-back reader.GetX(n) ordinals — both correct), DarlingManagedRoles/provision-roles.sql (remediation password correctly excluded from the viewer's SELECT grant and classified as a secret column), and the OperatorRemediationGate/OperatorRemediationFlow state machine logic and its tests. No SQL injection risk (all values parameterized), no secrets logged (WarnIfRemediationCredentialIsInert logs only which half is missing, never values), and the "monitoring credential never used for a write" invariant is enforced structurally (separate connection-string builder, separate resolver, no fallback).

One finding — stale doc comment (not inline-commentable; outside the diff hunk)

Darling/PerformanceMonitor.Darling.Service/PgPlanForceActionStore.cs, line 279 (in GetPendingReviewsAsync's XML doc, just above the method):

OWN-FORCES-ONLY is structural here — the read starts from rows this bot journaled, so an operator's hand-placed force can never surface as something to unforce.

This line was left unchanged by the PR, but it's exactly the claim the PR invalidates: V113 makes the operator a second writer to collect.plan_force_actions, so the property is no longer structural — it's now enforced by the explicit AND pfa.actor = 'bot' predicate added a few lines later in the same method's query body. The new SQL comment right there even says so directly: "Until phase 1 this read's comment could say the property was structural because the bot was the only writer to the table." That's a correct description of the old doc comment above it, but the old comment itself wasn't updated to match. Worth a one-line fix so the XML summary doesn't contradict the query it's describing (e.g. "OWN-FORCES-ONLY is enforced by the actor = 'bot' filter below, not structural — see V113").

Everything else looked solid — no other correctness, parity, security, or performance issues found.

…V113

Five sites CI found, four of them the ratchets a new rung always trips and one
of them the rung working as designed.

The retention end-to-end's raw INSERT into plan_force_actions omitted actor and
raised 23502 against a live store. That is the dropped DEFAULT doing its job: an
INSERT that forgets the actor must fail rather than silently claim to be the bot,
because a bot row is the kind the self-review may unforce.

V112's two suites asserted they were the top rung, which V113 makes false. That
claim belongs to whichever rung actually is top, or every new rung breaks every
older rung's suite, so it moved here: the all-sentinels-true mapping, the
last-ordinal equality, and the one-rung-behind check. V112 keeps its own ordinal
and now switches off every LATER sentinel for its behind-check, which generalises.

The Lite divergence pin's schema half was over-broad for the second time in one
file: plan_correction carries a generated last_good_plan_force_failure_reason
column, so a statement-substring scan reported the monitored server's own Query
Store forcing-failure reason as a bot journal. It now parses table NAMES, with
plan_correction as the control proving the parse discriminates them.
@claude

claude Bot commented Sep 8, 2026

Copy link
Copy Markdown

Reviewed the full diff (credential seam, actor column/journal filter, OperatorRemediationFlow/OperatorRemediationGate, viewer probe rung, Lite divergence pin, provision-roles.sql).

Correctness: Traced every new positional-parameter INSERT/SELECT in StoreConfigProvider.cs and PgPlanForceActionStore.cs against their column lists — no off-by-one or misaligned bindings (a common failure mode when a migration inserts two new nullable columns into the middle of an existing seed INSERT). OperatorRemediationFlow.Observe's limb/verdict precedence (identity before cost, executions-limb-wins-on-tie, dead-band on both sides) matches its own test suite exactly, including the NaN/Infinity/negative-baseline edge cases and the 0x-prefix/case/whitespace plan-hash normalization.

Security: All new SQL is parameterized (AddWithValue/$n positional params), no string concatenation of user/config data into SQL text. The remediation credential correctly has no plaintext fallback and no auto-fallback to the read-only monitoring credential. remediation_encrypted_password is correctly added to the secret-column ACL in both DarlingManagedRoles.cs and the hand-maintained provision-roles.sql (and there's an existing drift test, ProvisionRolesAclDriftTests, that would catch the two going out of sync). The AlterServerStateProbeSql capability probe is a read-only SELECT/has_perms_by_name with no interpolation.

Lite/Darling parity: V113 is Darling-only by design (write path is deliberately unimplemented in this PR), and that's backed by OperatorRemediationLiteDivergencePinTests, which will fail loudly if Lite ever gains the credential columns, the journal, or the executor tokens without the scope decision being revisited. That's the right way to encode an intentional, temporary asymmetry.

Style: New T-SQL (migration DDL, the probe SELECT) follows CONTRIBUTING.md conventions (uppercase keywords, column_name = expression aliasing, AS on aliases). OPTION(RECOMPILE) doesn't apply here since nothing added is a parameterized collector query against a monitored server's variable-selectivity DMVs.

No correctness, security, or parity issues found. One purely-cosmetic observation, not a blocker: in OperatorRemediationFlow.Observe, limb == ObservationWindowLimb.Elapsed && observation.ObservedExecutions < minObservationExecutions — the right-hand clause is always true whenever limb == Elapsed, since Limb() only returns Elapsed when executions is already below the floor. Looks intentional (belt-and-suspenders, per the surrounding comment), so no action needed unless you want to simplify it to a plain limb == Elapsed check.

…t the rule is

The literal exclusion list is the mechanism, not an obstacle: an exclusion is
only legitimate with an argument, and the argument sits in the comment beside the
key, so growing the list has to appear in a diff. V113's two keys make it four.

Bumping a literal by reflex is how a guard like this stops meaning anything, so
the property behind it is now asserted too: only a credential-shaped key may be
excluded. Excluding trustServerCertificate — the field #2552 actually reported —
would fail that, where a bumped literal alone would have let it through.
@claude

claude Bot commented Sep 8, 2026

Copy link
Copy Markdown

Reviewed this against CONTRIBUTING.md's style rules and the Lite/Darling parity requirement. Summary:

Scope. This is scaffolding only, matching the PR description: the remediation credential seam, the V113 migration, the journal's actor column, and the pure OperatorRemediationFlow/OperatorRemediationGate state machines. Nothing in this branch calls the new gate/flow from production code and nothing opens a write path — IPlanForceExecutor stays unimplemented, confirmed by grep (only referenced from their own definition files and DarlingConfig.cs's doc comments). Risk surface is low.

Correctness. Traced the V113 migration's parameter binding in StoreConfigProvider.cs end to end ($1...$19 against the INSERT's column list and the corresponding AddWithValue call order) — it's correct despite the out-of-numeric-sequence placeholders ($14, then $16/$17 added after $15, then the new $18/$19). Same for the SELECT/BuildServerFromRow ordinal mapping and the MapProbedSchemaVersion sentinel-append (ProbeOrdinal = 88, verified against the actual parameter count). The GetPendingReviewsAsync predicate correctly parenthesizes the new actor = 'bot' filter alongside the existing outcome/orphan-intent OR-clause. The evict-then-observe state machine's limb precedence (executions checked before elapsed) and plan-identity-vs-cost evidence split are internally consistent and match their own test suite.

One doc/comment inconsistency — left as an inline comment on PgPlanForceActionStore.cs: GetPendingReviewsAsync's XML summary still asserts own-forces-only is "structural" (pre-V113 wording), which now directly contradicts the SQL comment a few lines below it explaining that the same guarantee is enforced by the new actor = 'bot' predicate instead. Worth fixing since a stale claim like that is exactly the kind of thing this codebase's own comment discipline exists to prevent someone from trusting later.

Lite/Darling parity. This is Darling-only by design, and unusually for a cross-SKU change, the asymmetry is enforced rather than just described: OperatorRemediationLiteDivergencePinTests fails if Lite ever grows the credential, the executor seam, the journal, or a force/evict statement, and fails separately if Lite's stated advise-only rationale is ever deleted out from under it. No parity drift to flag.

Security. No injection surface — the new capability probe (has_perms_by_name(NULL, NULL, 'ALTER SERVER STATE')) is a hardcoded, parameterless SELECT. Credential storage follows the existing DPAPI-blob / env:/file: reference pattern with no plaintext remediation slot. BuildRemediationConnectionString correctly forces ApplicationIntent.ReadWrite regardless of the server's monitoring-only ReadOnlyIntent, and throws rather than silently degrading for a Postgres target or a missing credential.

Style. The V113 Postgres migration schema-qualifies every object (config.*, collect.*) per CONTRIBUTING's Darling rules; no T-SQL files are touched by this PR so the collector-specific rules (OPTION(RECOMPILE), etc.) don't apply here.

Review catch, and a correct one: the method's summary still said the property
held because the read starts from rows the bot journaled. That was true while the
bot was the table's only writer and V113 makes it false, so the summary
contradicted the SQL comment a few lines below it. A stale WHY-comment is worse
here than none — it invites removing the actor filter for looking redundant,
which is exactly the guarantee it now carries alone.
@claude

claude Bot commented Sep 8, 2026

Copy link
Copy Markdown

Review: #2138 phase 1 (remediation credential seam + journal actor)

Went through all 27 changed files (the V113 migration, the credential seam, OperatorRemediationFlow/OperatorRemediationGate, the connection-string/secret-resolution paths, the viewer probe wiring, ACL/provisioning parity, and the Lite divergence pin) against CONTRIBUTING's Darling migration checklist, T-SQL style, and Lite/Darling parity rules. No blocking issues found. Specific things I verified rather than took on faith:

Migration ladder (V113) — all four required parts are present and correct: rung registered at PgMigrations.Scripts[^1], StorageVersion.SchemaVersion = 113, both ALTER TABLE statements schema-qualified (config.*, collect.*), and the viewer probe (StoreSchemaProbeSql sentinel, the new reader.GetBoolean(88), the trailing hasRemediationCredentialAndActor = false parameter, and its if arm placed above V112's) all line up — confirmed the parameter count is 89 (ordinals 0–88), matching RemediationCredentialRungTests.ProbeOrdinal = 88. The DEFAULT 'bot'DROP DEFAULT ordering on actor is correct for an honest backfill that then fails closed on a forgotten column.

Positional parameter bindingStoreConfigProvider's seed INSERT is the kind of change that silently miscounts columns; walked through it by hand and the 23 columns / 23 values ($1$19 plus NULL, TRUE, FALSE, $15 reused) line up, with the two new AddNullableText calls landing at $18/$19 in the same order they're declared in the column list.

Own-forces-only invariantGetPendingReviewsAsync now filters actor = 'bot', GetQueryHistoryAsync deliberately does not (documented and correct: those aggregates should count operator rows too), and the doc comment that used to call the property "structural" was updated in the last commit to stop contradicting the SQL filter — good catch fixing that in-branch.

Gate/flow logic (OperatorRemediationGate.SurfaceFor, OperatorRemediationFlow.Observe) — pure, well-covered by the new test suites (window-limb precedence, the plan-identity-vs-cost evidence asymmetry, the asymmetric dead-band via MaterialChangeRatio vs its reciprocal, hash normalization). Nothing wired to an execution path yet, matching the PR description and confirmed by grep — BuildRemediationConnectionString/ResolveRemediationPassword/AlterServerStateProbeSql have no production caller outside tests.

No-write-path guarantee — checked that none of the new doc comments (MonitoredServerConnection.cs, IPlanForceExecutor.cs) leak sp_query_store_force_plan/FREEPROCCACHE as compiled string literals; they're all in /// doc comments, and darling.sample.json's mention of DBCC FREEPROCCACHE is a <None Include> copied file, not embedded — so PlanForceNoWritePathTests' assembly-bytes scan stays green for the reason it should.

ACL/provisioning parityDarlingManagedRoles.ViewerRestrictedConfigTables and provision-roles.sql's hand-mirrored GRANT list agree on both column set and order; remediation_encrypted_password is correctly on the secret side and not granted to viewer in either place.

Lite divergence — the new pin test justifies the Lite/Darling asymmetry explicitly (advise-only scope statements in LiteRecommendationItem/LiteRecommendationsViewModel) rather than just asserting it, and checks both that the capability is absent and that the scan can actually fire (positive control against the Darling journal). This is the right way to keep an intentional SKU divergence from silently drifting into an accidental one.

One trivial nit, not worth blocking on: darling.sample.json's new remediationUsername/remediationEncryptedPassword entries are shown as empty strings, while the comment above them says "Omit both keys (the default)". Functionally identical (HasRemediationCredential treats blank the same as absent), just a small mismatch between the example and its own stated guidance.

Comment thread PerformanceMonitor.Analysis/OperatorRemediationFlow.cs
@claude

claude Bot commented Sep 8, 2026

Copy link
Copy Markdown

Review

Went through the full diff (all 6 commits, ~3000 lines) against CONTRIBUTING.md's T-SQL style rules, Lite/Darling parity requirements, and general correctness/security.

Scope actually shipped, confirmed against the description: the credential seam (remediation_username/remediation_encrypted_password, V113), collect.plan_force_actions.actor plus the actor-filtered GetPendingReviewsAsync read, and the pure OperatorRemediationFlow/OperatorRemediationGate state machine. Confirmed there is genuinely no production caller wired to either new class yet (grep for OperatorRemediationFlow./OperatorRemediationGate. outside Tests turns up only XML-doc cross-references) — this really is seam-and-logic-only, matching the 'what is not in this branch' section.

Correctness

  • Traced every changed parameter list against its column list by hand: the V113 registry seed INSERT (StoreConfigProvider.SeedMonitoredServersAsync, now 19 params), the registry read SELECT/ordinal mapping, and the plan_force_actions journal INSERT/SELECT ordinal shifts from inserting actor at position 9. All positions line up correctly — no off-by-one from the new columns.
  • The V113 migration's add-default-then-drop-default sequencing for actor is correct and matches the stated backfill intent; it's also directly pinned by RemediationCredentialRungTests.TheActorDefaultIsAddedForTheBackfillThenDroppedSoAForgottenInsertFails.
  • Left one inline question on OperatorRemediationFlow.Observe about whether the plan-identity check should be gated behind the observation window the same way the cost check is — the class's own reasoning says identity needs only one compile to settle, but the code doesn't act on a hash match until a limb closes. Not a live bug today (no caller exists yet), but worth settling before phase 2 wires an executor to it.

Lite/Darling parity

  • This PR is intentionally Darling-only, and the divergence is unusually well-guarded: OperatorRemediationLiteDivergencePinTests asserts both that Lite has none of the new capability tokens and that the advise-only scope statements the decision rests on still exist in Lite's source, so the pin can't quietly outlive its own justification. That satisfies the parity-drift concern here — flagging per the instructions, but no action needed.

Security

  • No SQL injection surface: all new SQL is either static DDL (migrations) or fully parameterized (Npgsql $n params via AddWithValue/AddNullableText).
  • remediation_encrypted_password is correctly added to the viewer's SecretColumns list (and not granted in provision-roles.sql), while remediation_username is correctly non-secret and granted — both directions pinned by RemediationCredentialRungTests.TheCredentialColumnsAreClassifiedWithTheSecretOnTheSecretSide and DarlingManagedRolesTests.
  • The new AlterServerStateProbeSql is a parameterless, literal SELECT with no write keywords, matches the column_name = expression alias style from CONTRIBUTING.md, and is pinned read-only by TheCapabilityProbeIsReadOnly (which itself has a positive control against a real write statement — good, avoids a scan that guards nothing).
  • ResolveRemediationPassword/BuildRemediationConnectionString correctly require both credential halves, never fall back to the read-only monitoring credential, and force ApplicationIntent.ReadWrite so a readOnlyIntent hint on the monitoring entry can't misroute a future write.

Style

  • New T-SQL (the probe) and the Postgres migration SQL follow the documented conventions (uppercase keywords, schema-qualified objects, column_name = expression aliasing). No missing-index-DMV suggestions applicable here.

No blocking findings. One open design question posted inline on OperatorRemediationFlow.cs.

…cluded

Review asked whether the identity arm waiting for a limb to close was deliberate.
It was not: the window guard went first as the natural "don't decide yet" and the
hash-already-matches case was never considered. Nothing pinned it either way, so
it read as an open question. It is now a decision, and the behaviour is unchanged.

The wait is right, but for a reason the class did not give. FREEPROCCACHE evicts
the plan CACHE and Query Store keeps its plan row, so the regressed hash is
present the instant after the eviction and stays present. ActivePlanHash only
becomes evidence once post-eviction executions have been attributed to a plan, so
an identity arm running while the window is open would not catch an early
recompile — it would read the pre-eviction plan and offer a force on it, on the
first call, every time. Query Store's default 900-second flush interval is also
why the elapsed limb's 30 minutes is an order of magnitude rather than a round
number.

Second reason, independent of the instrument: OptimizerRecovered needs the
executions floor, so firing identity earlier would make the force the quick
answer and "nothing here needs pinning" the slow one. Wrong asymmetry for a lever
whose premise is that the cheapest fix pins nothing.

The class's own "one compile settles it" was the overstatement that made this
look accidental — true of the optimizer, false of the instrument. Corrected here
along with ActivePlanHash's contract, whose old wording invited deriving the value
from plan-row presence, which after an eviction is always the regressed hash.
Comment thread Darling/PerformanceMonitor.Darling.Service/darling.sample.json
@claude

claude Bot commented Sep 8, 2026

Copy link
Copy Markdown

Reviewed the diff (production code + migration + tests) against CONTRIBUTING.md's T-SQL/C# style rules, Lite/Darling parity, and the credential-handling security surface. This is a Darling-only, no-write-path PR (correctly — IPlanForceExecutor stays unimplemented), so Lite/Darling parity is enforced by the new OperatorRemediationLiteDivergencePinTests, which is doing its job (verified the file paths/phrases it reads against actual Lite/ source — they match).

Checked and correct:

  • V113Sql schema-qualifies every object (config.config_monitored_servers, collect.plan_force_actions) per the migration-rung rule.
  • StoreConfigProvider's SeedMonitoredServersAsync INSERT — the new $18/$19 parameters line up positionally with remediation_username/remediation_encrypted_password in both the column list and the AddWithValue call sequence; the added-then-dropped DEFAULT 'bot' behaves as documented (backfills existing rows, then makes a forgetful INSERT fail loudly — the retention test exercises exactly that 23502).
  • PgPlanForceActionStore's two SELECT sites (GetPendingReviewsAsync, GetRecentActionsAsync) both added pfa.actor at the same ordinal and RecordFrom's reader-ordinal mapping shifted consistently across both callers.
  • provision-roles.sql grants only remediation_username (non-secret) to viewer, not remediation_encrypted_password — matches DarlingManagedRoles' secret-column classification.
  • BuildRemediationConnectionString's ApplicationIntent = ReadWrite override (never routes a write through a read-only-intent hint) and its separate ApplicationName are both correctly reasoned and don't touch the read path.
  • No secret material (password/blob contents) is logged anywhere in the new code — WarnIfRemediationCredentialIsInert only logs presence booleans.
  • ViewerDataService.MapProbedSchemaVersion's new arm and the viewer probe SQL are correctly wired (89 sentinels, ordinal 88, verified against RemediationCredentialRungTests).

One gap (left as an inline comment on darling.sample.json): the new remediationUsername/remediationEncryptedPassword keys are real, user-settable darling.json config today, but Darling/README.md's per-server settings table doesn't document them, unlike every sibling key.

No correctness bugs, no injection surface (all params bound, migration SQL has no interpolated identifiers), no missing-index-DMV suggestions here. The pure OperatorRemediationFlow/OperatorRemediationGate decision logic checks out against its own stated invariants (dead-band math, limb precedence, nullable-surface-not-a-flag).

Per the ruling on #2138: remediation execution is Darling-only, permanently. The
rule itself does not change — what changes is which of its two stated reasons
carries it.

The obsolete half named Dashboard as the owner of remediation execution, which
points at nothing live. The half that decides it is Lite's store: a DuckDB file is
per-workstation, so a remediation journal kept there cannot be the shared audit
trail such a journal exists to be — two operators acting on one server would each
hold half of it and neither could see the other's forces — and it cannot carry the
bot's own-forces-only invariant, which is a predicate plus a self-reference within
ONE shared store. No deprecation can invalidate that.

All three sites that state the rule now state that reason; the two that carried
the dead one no longer do.

The divergence pin is strengthened rather than relaxed: it keeps every rule phrase
it already asserted, adds the durable reason in all three files, and adds that the
dead reason is absent. The #2138 wording fix is the case that shows why rule and
reason need separate assertions — the rule never changed and one of its reasons
died, so a pin checking only the rule would have gone on passing over a
justification naming a deprecated SKU as the owner of a capability.

The Dashboard mentions in LiteRecommendationsViewModel and RecommendationsTab stay.
They are UI comparisons about an "Open in Active Queries" deep-link, not scope
claims, and their reason is the surface's tab shape, which the deprecation does not
touch. They now also serve as the positive control proving the dead-reason absence
assertions are not passing against a failed read.
@claude

claude Bot commented Sep 8, 2026

Copy link
Copy Markdown

Reviewed the diff against CONTRIBUTING.md conventions, Lite/Darling parity, and the credential/security surface. No correctness, security, or parity bugs found. Notes from the pass:

Credential seam (DarlingSecrets.cs, MonitoredServerConnection.cs, DarlingConfig.cs)ResolveRemediationPassword correctly returns null (not throw) for an unarmed server while ResolvePassword still throws for a misconfigured monitoring credential; the asymmetry matches the stated intent. BuildRemediationConnectionString correctly rejects Postgres targets and servers lacking HasRemediationCredential, forces ApplicationIntent.ReadWrite regardless of the monitoring entry's ReadOnlyIntent, and uses a distinct ApplicationName. Confirmed neither BuildRemediationConnectionString nor ResolveRemediationPassword has any non-test caller yet — matches the PR's "no write path ships" claim.

Migration V113 (PgMigrations.cs) — schema-qualified (config.*, collect.*), ADD COLUMN ... DEFAULT 'bot' then DROP DEFAULT is the right sequence to backfill existing rows while forcing new inserts to specify actor. New index idx_plan_force_actions_actor(server_id, actor, action, action_time) matches the GetPendingReviewsAsync predicate shape.

Parameter/ordinal bookkeeping — hand-checked the INSERT/SELECT column lists against the positional $n params and reader ordinals in StoreConfigProvider.cs and PgPlanForceActionStore.cs (seed INSERT, read-back SELECT, JournalAsync, GetPendingReviewsAsync, GetRecentActionsAsync) — all correctly shifted for the new remediation_username/remediation_encrypted_password/actor columns, no off-by-one.

own-forces-only invariantGetPendingReviewsAsync now filters actor = 'bot'; GetQueryHistoryAsync deliberately does not, with the asymmetry documented. PlanForceBot.BuildRecord stamps Actor: ActorBot unconditionally. Test coverage (PlanForceActionStoreTests) journals both an operator and a bot force and asserts the review read excludes the operator's row while the audit read (GetRecentActionsAsync) still returns both — good discriminating control.

Role grants (DarlingManagedRoles.cs, provision-roles.sql)remediation_username correctly classified non-secret and granted to viewer; remediation_encrypted_password added to SecretColumns and excluded from both the managed-role carve and the bring-your-own-Postgres script. The mcp role reuses BuildViewerColumnAclSql, so it inherits the same carve without needing a separate edit.

Lite/Darling parity — no Lite production code gained a credential, executor, or journal table (correctly, per the PR's stated Lite-is-permanently-advise-only decision). The Lite doc-comment updates in LiteRecommendationItem.cs/LiteRecommendationsViewModel.cs/RecommendationsTab.xaml.cs are comment-only and consistent with the new reasoning (DuckDB is per-workstation, can't hold a shared audit trail).

State machine (OperatorRemediationFlow.cs) — window guard is checked before the plan-identity comparison in Observe, so a matching hash can't short-circuit an open window (pinned by AMatchingHashWhileTheWindowIsStillOpen_IsNotYetAVerdict). SameHash correctly treats absent hashes on either side as non-matching and normalizes 0x prefix/case. Dead-band and floor/ceiling math check out against the paired tests.

Nothing here rises to a blocking finding — this is a clean, thoroughly self-tested PR.

Review catch. Darling/README.md's per-server table lists every other key in
darling.json with its default and notes, and CONTRIBUTING names that README as
the operator-facing configuration document — so two keys that are real, settable
config today were undiscoverable from it.

Both rows added, plus a paragraph for the part that does not fit a table cell: the
grants each lever needs, that ALTER SERVER STATE is genuinely optional because
evict-first degrades to force-only with a named reason, and that the service asks
the server with a read-only has_perms_by_name check rather than guessing per
platform. And a note that this release ships no consumer of the credential, so an
operator provisioning ahead of the write path knows the arming is inert for now
rather than discovering it silently does nothing.
@claude

claude Bot commented Sep 8, 2026

Copy link
Copy Markdown

Review: #2138 phase 1 (remediation credential seam + journal actor)

Went through all 9 commits/diff hunks (credential seam, OperatorRemediationFlow/OperatorRemediationGate, V113 migration, viewer probe, role ACLs, Lite divergence pin) plus cross-checked the areas most likely to hide a real bug in a change like this:

  • Parameter/column ordinal wiringSeedMonitoredServersAsync's INSERT (19 params, $18/$19 for the new columns) and BuildServerFromRow's SELECT (ordinals 17/18) both line up correctly against the new remediation_username/remediation_encrypted_password columns. Verified by hand rather than trusting the diff context.
  • ViewerDataService.MapProbedSchemaVersion arity/ordinal — counted the actual parameter list in the checked-out file: 89 bool has* params, hasRemediationCredentialAndActor is the 89th (ordinal 88), matching RemediationCredentialRungTests.ProbeOrdinal. No off-by-one.
  • OperatorRemediationFlow.Observe — the two-limb (executions vs. elapsed) split and the "plan identity needs fewer executions than a cost verdict, but not zero" fix from the earlier review round both check out; the window-still-open guard correctly gates the hash-identity arm now, and Limb() always resolves Executions over Elapsed when both are satisfied.
  • Own-forces-only invariantGetPendingReviewsAsync now filters actor = 'bot' while GetQueryHistoryAsync's restraint aggregates deliberately don't, and both are backed by a test that journals an operator-force/bot-force pair rather than trusting an empty-filter false positive.
  • Credential handling — no fallback from the monitoring credential, both halves required or it's unarmed (checked via HasRemediationCredential, not just the blob), ResolveRemediationPassword returns null rather than throwing for the (default) unarmed case, BuildRemediationConnectionString throws loudly for Postgres targets and for a missing credential rather than silently building a bad connection string. No secret ever reaches a log line I could find.
  • provision-roles.sql / DarlingManagedRoles parityremediation_encrypted_password is on the secret side, remediation_username on the non-secret side in both the C# ACL and the shipped SQL grant, and ProvisionRolesAclDriftTests (a pre-existing generic parser, untouched by this PR) will catch drift between them automatically.
  • Lite/Darling parity — this phase is deliberately Darling-only, and that's enforced by OperatorRemediationLiteDivergencePinTests (capability-token scan + schema-table scan + rule/reason source-text pins), not just asserted in prose. Nothing here silently reads as an empty feature on the Lite side.
  • Migration safety — V113's ADD COLUMN ... DEFAULT 'bot' then DROP DEFAULT ordering is correct and is itself pinned by position (not just presence); the retention e2e test's raw INSERT was updated to include actor with the right column/value count (11/11).

No correctness, security, or Lite/Darling parity issues found. T-SQL style (AS aliases, column_name = expression, e.g. AlterServerStateProbeSql) looks consistent with CONTRIBUTING.md; that probe is read-only, unused by any production call site yet (it's plumbing for a later phase, which the PR description is explicit about), and its own test (TheCapabilityProbeIsReadOnly) guards against it silently growing a write.

Nothing to flag as a blocking or non-blocking finding.

@claude

claude Bot commented Sep 8, 2026

Copy link
Copy Markdown

Reviewed the diff (production code, migration, and tests) against CONTRIBUTING.md's Postgres/T-SQL conventions, Lite/Darling parity, security, and correctness.

No blocking issues found. This is an unusually well-instrumented change:

  • Credential seam (DarlingSecrets.ResolveRemediationPassword, MonitoredServerConnection.BuildRemediationConnectionString): correctly never falls back to the monitoring credential, requires both halves before treating a server as armed, uses a distinct ApplicationName, forces ApplicationIntent.ReadWrite, and throws loudly for Postgres targets and unarmed servers rather than silently degrading.
  • Migration V113 (PgMigrations.cs): schema-qualifies every object (config.*, collect.*) per the Postgres style rules, uses the add-default-then-drop-default pattern correctly to backfill existing rows as 'bot' while making a future omitted actor fail loudly (23502), and adds a purpose-built index for the new (server_id, actor, action, action_time) review-read shape rather than piggybacking on the existing time index.
  • Parameter wiring: traced the StoreConfigProvider INSERT/SELECT param ordinals by hand ($1..$19 against the AddWithValue call order, and the SELECT column list against the reader ordinals in BuildServerFromRow) — both line up correctly with the two new columns appended at the end.
  • OperatorRemediationFlow/OperatorRemediationGate: pure, well-covered decision logic; the window-limb precedence, the plan-identity-vs-cost evidence asymmetry, and the "both verdict halves must agree or refuse" gate logic all check out against their test suites.
  • Security: all queries are parameterized, no secrets logged, remediation_encrypted_password is correctly classified as secret and excluded from the viewer role grant in provision-roles.sql, and the new has_perms_by_name capability probe is genuinely read-only (verified by hand against the DDL-keyword list its own test scans for).
  • Lite/Darling parity: the Darling-only scope decision is deliberate and enforced by OperatorRemediationLiteDivergencePinTests, which asserts both the absence of the capability in Lite and that the stated rationale (per-workstation DuckDB store can't be a shared audit trail) still exists in Lite's source — this is exactly the kind of parity drift this repo should be watching for, and it's already pinned.
  • Confirmed no write path ships in this PR (no sp_query_store_force_plan/unforce_plan/FREEPROCCACHE execution reachable from the new code), matching the PR description's claim.

Left one non-blocking inline nit on DarlingServerConnector.cs about the inert-credential log message also firing for PostgreSQL targets, which can't use this feature at all per BuildRemediationConnectionString.

erikdarlingdata and others added 3 commits September 8, 2026 11:38
…Postgres

ResolveConnectionString runs on both engine paths, so the inert-credential notice fired
for PostgreSQL targets too and told the operator nothing would use it YET. On Postgres
it is inert permanently: BuildRemediationConnectionString throws for that engine and
plan-force remediation forces a Query Store plan. One message for both promised a
future that engine does not have.
@claude

claude Bot commented Sep 8, 2026

Copy link
Copy Markdown

Reviewed the full diff (production code + tests) against the correctness / Lite-Darling-parity / security / performance / T-SQL-style checklist.

Summary: no blocking issues found. This is an unusually disciplined change — worth calling out what makes it hold up rather than just saying "LGTM":

  • Credential seam (DarlingSecrets.ResolveRemediationPassword, MonitoredServerConnection.BuildRemediationConnectionString): null-on-unarmed vs throw-on-misconfigured is the right asymmetry, both-or-nothing arming (HasRemediationCredential) avoids decrypting a secret that turns out to be unusable, and the credential never round-trips through the monitoring connection builder. remediation_encrypted_password is correctly classified as secret in DarlingManagedRoles and correctly not granted to viewer in provision-roles.sql (only remediation_username is) — checked both stay in sync with the C# ACL list order.
  • V113 migration: schema-qualified (config.*, collect.*), the add-default-then-drop-default sequence on actor is correct for backfill-then-enforce, and the connect-time gate (ViewerDataService.MapProbedSchemaVersion) probes a column that's actually new (table existence can't discriminate since both tables predate this rung) — verified the sentinel ordinal/arity bookkeeping is internally consistent with the ladder-census tests.
  • actor filtering asymmetry (GetPendingReviewsAsync filters actor = 'bot', GetQueryHistoryAsync deliberately doesn't): reasoning holds — one authorizes an undo (must stay bot-only), the other restrains the bot's own future actions (correctly counts operator activity too).
  • OperatorRemediationFlow.Observe / OperatorRemediationGate.SurfaceFor: pure functions, exhaustively tested over the verdict/limb/capability state spaces (including the disagreement-refuses and never-silent-degrade invariants). Traced the executions-floor vs. elapsed-timeout precedence and the plan-identity-vs-cost evidence asymmetry by hand — didn't find a gap the tests don't already cover.
  • Lite/Darling parity: Lite gets no credential fields, no journal table, no executor seam — confirmed via the new divergence pin test's token scan and independently via grep across Lite/. The three doc-comment sites restating "advise-only" were updated consistently (store-based reasoning replacing the dead deprecated-SKU reasoning) in all three files.
  • T-SQL/SQL style: migration SQL is schema-qualified with trailing commas and uppercase keywords where applicable; AlterServerStateProbeSql has no OPTION(RECOMPILE), but that's consistent with this codebase's existing one-shot connect-time probes (e.g. DetectionQueryText), not a collector query with variable selectivity, so not a deviation.

Didn't flag any inline comments — everything I traced through (parameter ordering in the StoreConfigProvider seed INSERT, the PgPlanForceActionStore reader column-index shifts after adding actor, the drift-exclusion regex) checked out against the surrounding code.

@erikdarlingdata
erikdarlingdata merged commit b8ae88a into dev Sep 8, 2026
8 checks passed
@erikdarlingdata
erikdarlingdata deleted the feature/2138-phase1-remediation branch September 8, 2026 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant