Repository navigation
TOOL_STRATEGY_RULES conflates safety policy with tool routing; split it and delete routing in favor of skill/schema-carried routing #1820
Description
Overview
TOOL_STRATEGY_RULES (packages/agent/src/agent_guidance.rs:35-58) states its purpose as "the search-first mandate, tool selection hints, and canonical create/update/connect patterns." Read line by line, it is doing four unrelated jobs, and only one of them belongs in resident prose under ADR-064.
Surfaced while investigating #1814: the core-side confirmation found update_node never reaches search_skills at all, because line 48 hardcodes its routing in prose — and the Graph Editing skill already carries the correct "find then update" instruction, unreachable behind the prose that preempts it.
The four categories, and what each should be
1. Safety invariants (lines 37, 38, 39, 40, 44) — belongs here, keep. No tools for chitchat; don't fabricate tool-call history; don't assert unverified schema facts; don't repeat identical calls; don't claim an action without a result. Genuine policy with no other home. This is the entire legitimate scope of the block under ADR-064 rule 5.
2. Argument-shape rule (line 36) — wrong channel. NODE_TYPE COMES FROM RELEVANT ENTITY TYPES governs argument construction. ADR-064 rule 1 assigns this to tool schemas.
3. Tool-usage reference (lines 50, 51, 52, 53, 56, 57) — wrong channel, and duplicate. How search_nodes filters work, when to prefer search_semantic, how to read a search_semantic result, how get_node/get_related_nodes work. This is API documentation in prose for tools that already carry their own descriptions — the same duplication class as #1809's coreValues bug, spread across more lines.
4. Per-operation routing (lines 41, 42, 48, 54, 55) — wrong channel, confirmed harmful. Each line tells the model which sequence of tool calls to make for a given request shape:
- Line 41: for
create_node, callsearch_skillsthencreate_node - Line 42: for
create_schema, skipsearch_skills - Line 48: always call
search_nodesbeforeupdate_node— measured to make the model bypasssearch_skillsentirely for this operation, in every trial of a controlled confirmation run - Line 54: for indirect updates, call
resolve_querythensearch_nodesthenupdate_node - Line 55: for
create_node, callsearch_skillsfirst forschema_metadata
Routing is exactly what ADR-038 assigns to skill-mediated discovery, not resident prose. Line 48 is now measured to actively defeat it: Graph Editing's own skill instructions say "FIND THEN UPDATE: call search_semantic or search_nodes first" — the correct routing, sitting dead behind the prose line that preempts it before the model ever reaches search_skills.
Evidence
packages/agent/src/agent_guidance.rs:31-34— the block's own stated purpose, which does not match categories 2–4- Core-side confirmation (Resolve whether skill instructions delivered as a tool result suppress action #1814): with real
LocalAgentLoop, real seeded skill content, and real tool schemas,update_nodecalledsearch_nodesdirectly in 100% of trials across both arms of a controlled test —search_skillswas never invoked, confirmed via full tool-call dump on sampled trials, not a scoring artifact Graph Editing's seeded guidance already contains the correct routing instruction for this operation- ADR-064 rule 1 (tool schemas own argument shape), rule 2 (tool descriptions own tool selection), rule 5 (resident prose owns identity and policy only)
Scope of this issue
Not "delete the routing lines and see what breaks." Each deleted line needs its routing decision confirmed present elsewhere before removal:
- Line 48 (
update_nodesearch-first): confirmed present inGraph Editing's skill instructions — verify it's sufficient once line 48 no longer preempts it - Line 41 / 55 (
create_node→search_skillssequencing): verify the equivalent skill's instructions carry this, or that it's redundant with the skill match itself - Line 42 (
create_schemaskipssearch_skills): verify this is expressed as a tool description or is structurally true (no skill exists to route through) - Line 54 (
resolve_query→search_nodes→update_nodefor indirect updates): dense, multi-step, not yet confirmed to have skill or schema coverage — check before deleting - Line 36 (node_type argument shape): move to the relevant tool schemas' parameter descriptions
- Lines 50-53, 56-57 (tool-usage reference): confirm each fact is already present in
search_nodes/search_semantic/get_node/get_related_nodes/create_relationship's own tool descriptions; add what's missing there, then delete from prose - Lines 37-40, 44 retained as the block's entire remaining content
- Regression coverage for each deleted routing line, exercised through the real skill-routing path, not the old hardcoded path
- Assembled prompt character count recorded before/after
Acceptance Criteria
- Every routing/reference line removed only after its equivalent is confirmed to exist in a tool schema, tool description, or skill's instructions — not removed speculatively
-
TOOL_STRATEGY_RULEScontains only safety-invariant policy when done -
update_nodeandcreate_nodeboth route throughsearch_skillswhen it is expected to fire, verified with the realLocalAgentLoop - No regression in the agent test suite
Architecture Impact
Implements ADR-064 rules 1, 2, and 5 against a single concrete file. Likely a prerequisite for #1815 (ADR-038 Stage-1/Stage-2 gates) — building model-judged routing on top of prose that already hardcodes routing for some operations means the two systems will disagree, with prose currently winning per the measured pattern.
Related: #1809 (the same channel-duplication pattern — resident prose contradicting a correct authoritative source — found in a different file). #1814 (the confirmation run that surfaced line 48's effect; that issue's scope is retrieval mismatch, not this routing-duplication problem, so it is filed separately rather than folded in).
Activity
malibio commentedon Jul 30, 2026
Frontend baseline: 171 test files passed, 4112 tests passed (46.20s)
malibio commentedon Jul 30, 2026
Caveat from #1814's create_node confirmation, worth weighing here
#1814's core-side confirmation for
create_nodefound a 100-point separation (control 38/38, mismatch 0/30) between a correctly-matched skill and a mismatched one — but the failure mode is relevant to this issue's scope, not just #1814's.All 30 mismatch trials: the model called
search_skills, received Bulk Import's instructions (document-import guidance — "pass the markdown content directly," nothing about a properties map), then calledcreate_nodewith the rightnode_typebutcontentas a freeform sentence andproperties: {}— every field the user supplied silently dropped, turn reported as success. Not narration, not refusal — silent data loss with no signal anything went wrong.The open question, not yet isolated: whether this specifically requires a mismatched skill, or whether any skill whose instructions omit field-shape guidance produces the same result once it happens to be the one returned. #1814's run didn't run a third arm to separate those two.
That distinction matters for this issue's fix, not just #1814's:
TOOL_STRATEGY_RULESline has equivalent coverage in a skill/schema before removal) needs to check not just that a skill exists for an operation, but that its instructions actually specify the properties/field contract — not just "call this tool" routing. A skill that only says "call create_node" without Node Creation's "REQUIRED FIELDS... MUST be in the properties map" language would reproduce this failure even with perfect retrieval.Worth keeping in mind when auditing lines 41/55 (
create_noderouting) specifically: confirming the model reachessearch_skillsisn't sufficient on its own — the skill it lands on also needs to carry the field-shape contract, or removing the prose routing line succeeds at routing while leaving this silent-drop failure mode intact.Not proposing new scope here — flagging it so it's not lost, and so "the skill already covers this" isn't accepted at face value during the audit without checking what the skill's instructions actually specify.
TOOL_STRATEGY_RULES conflates safety policy with tool routing; split …
Record the honest post-ADR-064 baseline: 9/12 (6/9/8 across 3 runs), …
Make the agent eval diagnostic before recording a baseline; revisit A…