Skip to content

get_analysis_findings/get_analysis_facts move reading guidance off tools/list (#3898) - #4083

Merged
erikdarlingdata merged 3 commits into
devfrom
feature/3898-content-analysis
Sep 23, 2026
Merged

erikdarlingdata merged 3 commits into
devfrom
feature/3898-content-analysis

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

Refs #3898.

Why

Darling's tools/list cost about 330 KB before a client asked its first question. get_analysis_facts and get_analysis_findings each carried a long description with reading guidance mixed into it. This PR moves that guidance off tools/list and into get_tool_guide, which serves it on request. Nothing is deleted, only moved.

What changes

Both tools convert on Darling (Darling/PerformanceMonitor.Darling.Service/Mcp/DarlingMcpTools.cs) and Lite (Lite/Mcp/McpAnalysisTools.cs). Their original descriptions were already byte-identical between the two products, so the new heads and tails are byte-identical too.

get_analysis_facts's head states:

  • it returns raw scored facts, not findings.
  • an ANOMALY_* fact can appear here even though it stayed under the finding floor. Its presence does not mean a finding fired.
  • baseline_confidence (a fact's own baseline trustworthiness) is a different number from a finding's confidence in analyze_server.
  • collection_caveats is absent on a clean read. When present, it names families that were not read. Their absence from the facts is not evidence they were quiet.
  • ranked is absent unless 2 or more objects ride.

get_analysis_findings's head states:

  • these are persisted findings from past runs, not a new analysis, deduplicated to the latest occurrence per chain plus occurrence stats.
  • an empty answer is a true zero. Run analyze_server for new findings.
  • truncated (see truncation_note) flags a window-covering read cap. When it is set, occurrence stats can under-report.
  • confidence_basis flags rows persisted before this scoring existed as path-shape, not corroboration.
  • remediation_command/structured_remediation are advisory only. Neither is ever executed.
  • the three recurrence fields are labels at unchanged severity, not an escalation.

Before/after served characters

Tool Product Before After
get_analysis_facts Darling and Lite 1,825 585
get_analysis_findings Darling and Lite 2,875 610

Parameter trims (D2)

Converting a tool caps its parameter descriptions at 200 characters. Two needed a trim. The dropped detail rides on the tool's own tail, the same pattern compare_analysis.baseline_hours_back set in an earlier #3898 PR:

  • get_analysis_facts.source: 515 to 180 characters. The full 34-value accepted list moved to the tail, appended as source accepts: anomaly, bad_actor, .... The parameter description now states the guardrail instead: an unrecognized value is refused with the accepted list.
  • get_analysis_findings.include_drilldown: 235 to 122 characters. Two things moved to the tail, as an appended sentence. One is the two examples: parameter-sensitive plans, top spill queries. The other is the reason it defaults off: rows can be bulky.

A gap I filled while I was in this tool (small, in-lane)

get_analysis_findings's payload has always carried truncated and truncation_note. I confirmed this against the method body. It is already tracked as a real wire field in McpPayloadContractCensusTests.CutNoteKeys. But the original description never mentioned either field.

A caller reading only the description had no way to know one thing. Occurrence stats (occurrences, first_seen) can under-report when a window holds more than the read cap. I was already rewriting this tool's head from scratch, so I added one sentence covering it. This is new coverage, not a correction of wrong text. It does not affect the "nothing deleted" claim below.

Corrections

None. Both tools' original prose was accurate against their method bodies (checked GetAnalysisFacts and GetAnalysisFindings on both products).

D4 removals

get_analysis_findings carried two issue references. Both are off the wire now, in the head and the tail:

get_analysis_facts carried no issue references.

Re-pointed pins

  • Darling/Darling.Tests/FactSourceRegistryTests.BothDescriptions_SpellOutTheRegistry: used to read the full accepted-source list off the source parameter's own description. Re-pointed to check the refusal guardrail on the parameter description, and the full list on get_analysis_facts's tail (McpToolGuideTests.Served("get_analysis_facts").Tail). Lite's half of this test reads the whole file's raw text and needed no change, since the list still appears in that file, just relocated.
  • Lite.Tests/McpMissMessageParityPinTests: one pinned cross-SKU sentence quoted (pre-#3538) verbatim. Updated the pinned string to match the new, ref-stripped wording. The sentence itself is unchanged.

D9 traps

  1. Q: get_analysis_facts lists an ANOMALY_* fact. Did that fact cause a finding in analyze_server?
    Correct: not necessarily. The fact can fire and still stay under the finding floor.
    Prevents it: the head's "even when they fired but stayed under the finding floor, so a fact here did not necessarily produce a finding."
  2. Q: An ANOMALY_* fact shows baseline_confidence: 0.9. Is that the same number as a finding's confidence?
    Correct: no. One is the baseline's own trustworthiness. The other is a finding's evidence score.
    Prevents it: the head's "baseline_confidence ... differs from a finding's confidence in analyze_server."
  3. Q: get_analysis_facts returns collection_caveats naming the memory family. Does that mean memory was quiet?
    Correct: no evidence either way. That family was not read this pass.
    Prevents it: the head's "collection_caveats is absent on a clean read; when present, the families it names were NOT read, so their absence is not evidence they were quiet."
  4. Q: A fact in get_analysis_facts has no ranked field. Is that an error?
    Correct: no. ranked only appears when 2 or more objects ride the same fact.
    Prevents it: the head's "ranked is absent unless 2+ objects ride."
  5. Q: get_analysis_findings returns []. Does that mean the server is healthy?
    Correct: it means no findings were persisted in the requested window. Run analyze_server to check now.
    Prevents it: the head's "empty: no findings in the window, a true zero; run analyze_server for new ones."
  6. Q: A finding shows confidence: 1.0. Is that fully corroborated?
    Correct: check confidence_basis first. A row persisted before this scoring existed also reads 1.0, for a lone, uncorroborated symptom.
    Prevents it: the head's "confidence_basis flags rows persisted before this scoring existed as path-shape, not corroboration."
  7. Q: get_analysis_findings shows occurrences: 4 for a chain. Is that the true count for the window?
    Correct: check truncated first. If it is true, the window held more occurrences than the read cap, and occurrences/first_seen can under-report.
    Prevents it: the head's "truncated (see truncation_note) flags a read cap, so occurrence stats may under-report."
  8. Q: A finding carries remediation_command. Was it already run?
    Correct: no. It is advisory text for a human or agent to run deliberately. The read-only MCP never executes it.
    Prevents it: the head's "remediation_command/structured_remediation are advisory, never executed."
  9. Q: A finding shows recurring_at_this_hour: true. Is that a more severe finding than a one-off?
    Correct: no. It is a label at the same severity the pass already scored. A weekly problem is still just that problem.
    Prevents it: the head's "Recurrence fields are LABELS at unchanged severity."
  10. Q: get_analysis_facts was called with source: "waits" and returned shown: 0. Were there no wait facts at all?
    Correct: maybe not. total_facts (in the tail) can be nonzero. A filter or min_severity can bring shown to 0 while facts existed.
    Prevents it: nothing does, fully. See Handoff. total_facts/shown/filters were never documented before, and this PR does not add that.

Test plan

  • dotnet build Darling/Darling.Tests/Darling.Tests.csproj -c Debug: Build succeeded, 0 Warning(s), 0 Error(s).
  • dotnet build Lite.Tests/Lite.Tests.csproj -c Debug: Build succeeded, 0 Warning(s), 0 Error(s).
  • Targeted classes on both products (McpToolGuide*, McpToolsListBudget*, McpDescriptionTruthPin*, McpZeroIsAMeasurement*, McpPayloadContractCensus*, FactSourceRegistryTests, DarlingMcpToolsTests, McpMissMessageParityPinTests, McpAnalysisFindingsCommandTests): all green.
  • Full suite, Darling.Tests: Total 13,253, Failed 0, Skipped 631.
  • Full suite, Lite.Tests: Total 5,240, Failed 0, Skipped 0.
  • Dropped-sentence check (dropcheck.py): get_analysis_facts missing=0 on both products. get_analysis_findings missing=1 on both products. That one is the (pre-#3538) D4 removal listed above. The checker's ref-stripping regex only matches a bare (#nnn), not (pre-#nnn). No other sentence is missing.
  • Red-watch: reverted get_analysis_facts' ranked head sentence, confirmed McpToolGuideHeadsSqlCoreTests.EveryConvertedHead_CarriesItsGuardrailFact_AndThePointer failed, restored it, confirmed green again.
  • Plain-English checker on this body. The hits that remain are the D9 labels, quotes of the served head
    text, and the attribution emoji.

Coordinator verification

  • The coordinator merged origin/dev into the branch. The shared checkout was clean.
  • dropcheck: get_analysis_facts is missing=0 on both products. get_analysis_findings is missing=1 on both,
    and that one is the (pre-#3538) D4 removal. The literal confidence_basis value still starts with
    path-shape, and the head keeps the rule that such rows are not corroboration.
  • The two reworded parameters (source, include_drilldown) moved their extra text to each tool's tail.
  • D9: a fresh Haiku agent read only the served text of these two tools and answered the 10 traps. It misread
    none. Three answers (Q5, Q6, Q10) said "not stated", and a hedge is not a misread. Q10's Correct line says the
    tail documents total_facts. The original text did not, so the coordinator added one sentence to the tail on
    both products. It says that total_facts counts every scored fact before the source and min_severity filters. It also says
    that shown counts the facts that passed them, and filters echoes the two values. This matches
    DarlingMcpTools.cs lines 364-412. A new pin, FactsPayloadCounts_AreDocumentedInTheTail, holds the
    sentence in McpToolGuideHeads.SqlCore.cs on each product. It runs and passes on both.
  • Full suites after the merge and the coordinator commit: Darling.Tests Total 13260, Failed 0, Skipped 632.
    Lite.Tests Total 5243, Failed 0.
  • The Lite.Tests build gave one warning, xUnit1013 in QueryStoreDedupReadTests.cs. It came from Add exact module filtering to Query Store MCP results #4057, not
    from this PR, and The module-filter dedup test in QueryStoreDedupReadTests runs again (#4057) #4084 fixes it.
  • The CHANGELOG buffer entry now also says that the guide names what total_facts, shown and filters count.

Handoff

  • Done by the coordinator in this PR (commit acef622): get_analysis_facts' tail now says what total_facts,
    shown and filters count. See Coordinator verification.
  • Checked by the coordinator: _truncation_note is accurate. It is the drill-down's own key
    (DrillDownSerializer.TruncationNoteKey), separate from the top-level truncation_note field.
  • audit_config and mute_analysis_finding in the same two files are excluded from this PR by brief. They belong to a later lane in this family.

🤖 Generated with Claude Code

https://claude.ai/code/session_014nCo36doEcUWapnXbJvY7L

erikdarlingdata and others added 3 commits September 23, 2026 17:54
…ols/list (#3898)

Converts both tools on Darling and Lite (byte-identical original prose, byte-identical
new heads): tools/list now serves a short guardrail head plus a pointer to
get_tool_guide, which serves the full original prose as the tail.

- get_analysis_facts: 1825 -> 585 served chars.
- get_analysis_findings: 2875 -> 610 served chars.
- D2: trimmed the source and include_drilldown parameter descriptions to clear the
  200-char cap; the dropped detail rides on each tool's own tail.
- D4: dropped two issue refs from get_analysis_findings' tail ((pre-#3538), (#3653));
  re-pointed the one existing pin that quoted the ref text.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014nCo36doEcUWapnXbJvY7L
…unt (#3898)

The facts payload's total_facts, shown and filters fields were never documented. The tail now
says that total_facts counts every scored fact before the source and min_severity filters, shown
counts the facts that passed them, and filters echoes the two values. A tail pin on each product
holds the sentence.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014nCo36doEcUWapnXbJvY7L
@erikdarlingdata
erikdarlingdata marked this pull request as ready for review September 23, 2026 22:15
@erikdarlingdata
erikdarlingdata enabled auto-merge (squash) September 23, 2026 22:15
@erikdarlingdata
erikdarlingdata merged commit 79f53f1 into dev Sep 23, 2026
16 of 18 checks passed
@erikdarlingdata
erikdarlingdata deleted the feature/3898-content-analysis branch September 23, 2026 22:23
erikdarlingdata added a commit that referenced this pull request Sep 24, 2026
* Add 48 buffered CHANGELOG entries in one splice

Adds 48 entries and 48 link refs (#4057, #4077, #4079, #4081, #4082, #4083, #4084, #4085, #4086, #4087, #4088, #4089, #4090, #4091, #4092, #4093, #4095, #4096, #4099, #4100, #4101, #4103, #4105, #4106, #4107, #4108, #4109, #4110, #4111, #4113, #4114, #4115, #4116, #4117, #4118, #4119, #4120, #4121, #4122, #4123, #4124, #4125, #4126, #4127, #4131, #4133, #4136, #4141). Each PR's entry was buffered, and this lands every entry whose PR was merged on origin/dev when it ran.

Entries found under a bare section line (none unless the old script ran again) move into the matching ### section, below the new entries.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QQh8LczP1HGLXQYAx4oTZC

* Changelog splice: add #4137's entry, which merged before the splice but was missed

#4137 (pg_plan_capture reads csvlog) merged at 05:57Z with a CHANGELOG entry section in its
body. It was neither spliced nor listed as needing no entry. Its entry goes under Fixed, right
after #4136's, and #4136's last sentence now points to it instead of saying plan capture is
unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NUU29PuGg9TUFBsACgZg2K

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant