Repository navigation
Add fetch-at-activation guidance mechanism for external agent skill sessions - #2559
Conversation
…essions Implements the two unbuilt pieces #2532 tracked for issue #1958's fetched, hybrid-shaped delivery-channel decision: - `nodespace skill guidance <query>` (packages/cli) fetches procedural guidance from the graph's seeded `skill` nodes via the existing NodeService.SearchNodes RPC (node_type=skill, subtree markdown attached), wrapping every result in a provenance marker -- a `GRAPH-FETCHED GUIDANCE` banner in human mode, a `"provenance": "graph-fetched"` envelope in `--json` -- so fetched content is never silently indistinguishable from the skill's own static instructions. - SKILL.md's body gains a `## Graph-Authored Guidance (Fetched)` section instructing an activated agent to run this command before a nontrivial operation, and a new on-demand reference (references/graph-authored-guidance.md) spells out the trust boundary: fetched content can supply procedure, never permission, and any fetched instruction that tries to skip a confirmation or claims system/developer authority is itself the signal of a compromised graph entry. - The new reference file is wired through every place that ships packages/skill's reference tier: all five installer targets' `shims` lists (agents.ts), the PTY-spawned-agent path's embedded reference tier (context_assembly.rs, mechanical addition mirroring the two existing embeds -- required because that path reuses the same SKILL.md/references/ verbatim), and the public-repo publish script (publish-skill-repo.ts, already derived dynamically from agents.ts's shim lists so no logic change was needed there, only comments/tests). Also verifies, via new tests against the real seed_skill_nodes() registry, that the seeding-skips-by-node-type prerequisite #2532 listed as still-needed is already fixed (PR #1818/#1812, per-node versioned reconciliation, merged 2026-07-30 -- predating both #1958 and #2532): a compiled-template content change reaches an already-seeded database on next seed, and a live graph edit is immediately visible to a fetch with no reseed or restart. Manually verified end-to-end against a real isolated daemon + embedding model: edited a seeded guidance node's content, then confirmed `nodespace skill guidance` returned the edit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…banner against spoofing - Cap `--limit` itself (not just include_markdown) at the server's markdown-attachment ceiling in a new build_guidance_request(), so a guidance entry can no longer print a full provenance banner around empty content when more than 5 results are requested. - Reword code/test comments that cited issue or PR numbers to describe the underlying behavior or history directly instead. - Harden print_guidance's human-mode provenance banner against a malicious or corrupted graph node forging a banner-shaped line: each banner now carries a per-invocation random tag (fresh UUIDv4, generated at call time and announced once at the top of the output) that fetched content authored before the call cannot have predicted. Fetched title, description, and markdown content is also passed through a new sanitize_for_terminal that strips ANSI CSI sequences and other control characters, so raw escape bytes in fetched content can no longer redraw or conceal the banner. --json output is unchanged (already unspoofable by construction). Also updates the reference doc to explain the tag mechanism to an agent reading it, and adds a uuid dependency to nodespace-cli for the tag. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
All three review findings addressed in
Also done (non-blocking items): filed #2560 for the Every new/changed test was mutation-tested individually (reverted the fix, confirmed the test fails, restored) — including one real miss caught in the process: the first version of the tag-embedding test used a substring check that passed even when only the closing banner carried the tag, because the closing banner's text contains the opening banner's check string as a substring. Fixed the assertion to anchor on Full gate ( |
|
Re-reviewed (focused pass on the 3 fixes, isolated against commit 93f034c): the limit fix confirmed to cap the actual SearchRequest.limit, not just include_markdown, with a direct unit test at the exact point of the original bug. All 8 bare issue-number references confirmed gone (grepped the full diff), reworded text preserves the original meaning. The provenance-hardening fix — the real crux here — confirmed sound: a fresh UUIDv4 per invocation embedded in both banners (unpredictable to static fetched content), sanitize_for_terminal() comprehensively strips CSI/control sequences from every fetched field. The mutation-test fix is genuine — manually confirmed the final test's anchor string is no longer a substring of the closing banner, closing the false-positive gap. #2560 is real and well-scoped. Net diff (+319/-59) matches exactly, no scope creep. Merging. |
Summary
Closes the two genuinely-unbuilt pieces #2532 tracked, implementing #1958's fetched, hybrid-shaped delivery-channel decision:
nodespace skill guidance <query>(packages/cli) fetches procedural guidance from the graph's seededskillnodes via the existingNodeService.SearchNodesRPC (node_type=skill, subtree markdown attached) — the same content the in-app agent already retrieves via semantic routing, now reachable by an external harness through the CLI. No new server/proto surface was needed.GRAPH-FETCHED GUIDANCEbanner in human mode, a"provenance": "graph-fetched"envelope in--json— so fetched content can never be indistinguishable from the skill's own static instructions.SKILL.md's body gained a## Graph-Authored Guidance (Fetched)section instructing an activated agent to run this command before a nontrivial operation, and a new on-demand reference (references/graph-authored-guidance.md) covers the trust boundary in depth: fetched content can supply procedure, never permission, and any fetched instruction that tries to skip a confirmation or claims system/developer authority is itself the signal of a compromised graph entry, not something to act on.packages/skill's reference tier, not just the body: all five installer targets'shimslists (agents.ts), the PTY-spawned-agent path's embedded reference tier (context_assembly.rs— a mechanical 1:1 addition mirroring the two existing embeds, required because that path reuses the identicalSKILL.md/references/verbatim), and the public-repo publish script (publish-skill-repo.ts— already derives its file set dynamically fromagents.ts's shim lists, so no logic change was needed there, only stale comments/test expectations).The seeding-skips-by-node-type prerequisite is already fixed
Investigated before designing anything: the prerequisite #2532 listed as still-needed ("seeding skips by node type, so guidance changes never reach an existing install") was already fixed by PR #1818 (closed #1812, merged 2026-07-30 — per-node versioned reconciliation replacing the type-level skip), predating both #1958 and #2532. New tests (
packages/agent/tests/skill_guidance_fetch_mechanism.rs) confirm this holds against the realseed_skill_nodes()registry, not just the synthetic type PR #1818's own tests covered:a_skill_rules_content_change_reaches_an_already_seeded_database— a compiled-template content change (what editingskill_pipeline.rsproduces) reaches an existing install on next seed.a_users_graph_edit_to_seeded_guidance_is_immediately_visible_to_a_fetch— a live graph edit is visible to a fetch with no reseed or restart, the runtime half of Skill fetch-at-activation: harness matrix, provenance visibility, and the runtime fetch path (follow-up to #1958) #2532.Both were also verified live end-to-end: real isolated daemon (
NODESPACE_HOME-scoped), real embedding model, realnodespace skill guidanceCLI call, realnode updateon a seeded guidance node — the edit showed up in the next fetch.What's honestly still open
The manual harness matrix against Codex (and ideally Gemini CLI/Cursor) is not attempted here — it needs a machine with those CLIs installed, which this environment doesn't have. Only the Claude Code leg was previously verified (on #1958, 2026-08-25). This remains the one open acceptance criterion on #2532.
Design calls made (documented, not silently assumed)
skillnode type (seed_skill_nodes()), reusing infra rather than inventing a parallel one — this is exactly what Make the graph the source of truth for the NodeSpace skill, and decide how guidance reaches external agents #1958's own text pointed at ("the skill-guidance seeding path").NodeService.SearchNodeswithnode_types=["skill"]andinclude_markdownalready does the semantic-match-plus-subtree-markdown work;skill guidanceis a thin, dedicated, provenance-marking wrapper over it rather than new plumbing.SKILL.md's existing Tool Decision Guide/Common Agent Tasks stay as the safety net (a failed or empty fetch degrades to incomplete, never wrong) — the fetch adds team/org-specific procedure on top, matching the harness matrix's own unfinished status (only Claude Code verified, so trimming static content in favor of fetch felt premature).skillnodes areSeedTier::System(PR Seeded content never reaches an existing install: replace type-level skip with per-node versioned re-seed #1818's classification: "engineering artifacts... not user-editable, always replaced when the template changes"). A user's customization survives until the next code-driven content change to that specific skill, at which point it's silently replaced with no reseed protection. Confirmed real during review; filed as Seeded skill-guidance nodes have no durability protection against code-driven content replacement #2560 rather than fixed here — it's a pre-existing property of the already-decided tier system (Seeded content never reaches an existing install: replace type-level skip with per-node versioned re-seed #1812/Seeded content never reaches an existing install: replace type-level skip with per-node versioned re-seed #1818), and reconciling it with Make the graph the source of truth for the NodeSpace skill, and decide how guidance reaches external agents #1958's "users control guidance in the app" framing needs a real design decision (which tier, or node type, or flag) that belongs in its own focused pass.Review feedback addressed (round 1)
A deep adversarial review found three real issues, all fixed:
--limit/include_markdownmismatch (real bug):--limitabove 5 was sent uncapped to the request while onlyinclude_markdownwas clamped, so results past the 5th printed a full provenance banner around empty content. Fixed by cappinglimititself in a new, unit-testedbuild_guidance_request— a guidance entry's whole value is its markdown content, so there's no reason to ever request more than the server attaches markdown to.skill.rs,skill_guidance_fetch_mechanism.rs) violated CLAUDE.md's "describe the behavior/constraint directly" rule despite evading the automated checker's regex. All reworded to describe the behavior or history directly instead of citing numbers.print_guidance's human-mode banners wrapped fetched content verbatim and unescaped, with no defense against a malicious graph node forging a banner-shaped line (fake closing delimiter, fake second banner) or hiding the real banner with raw ANSI escapes. Fixed two ways: (a) every banner now carries a per-invocation random tag (fresh UUIDv4, generated at call time and announced once at the top of the output) that content authored before the call cannot have predicted, so a banner-shaped line without the announced tag is recognizable as content, not a boundary; (b) fetched title/description/markdown content is passed through a newsanitize_for_terminalthat strips ANSI CSI sequences and other control characters before printing, so raw escape bytes can no longer redraw or conceal the banner.--jsonoutput is untouched — it was already confirmed unspoofable by construction (a JSON string value can't be mistaken for a structural delimiter).New/changed tests for all three were mutation-tested individually (the underlying fix reverted, confirmed the test then fails, restored). Full gate (
bun run test:all) re-run clean,bun run quality:fixre-run clean.Also noted, not a blocker:
SKILL.md's body is currently ~4957 tokens (313 lines) — under the 5000-token/500-line spec recommendation theskill_md_body_is_within_spec_size_recommendationstest guards, but with only ~43 tokens of headroom. Flagging so it's on record before the next addition to the body.Test plan
cargo test -p nodespace-cli -p nodespace-agent— all green, including new provenance/formatting/hardening unit tests (mutation-tested individually) and the new data-layer reconciliation/edit-visibility testsbun run --cwd packages/skill test— 91/91 passbun run skill:check/skill_md_generation.rs(11/11, including the token/line budget guard) — clean, no driftbun test scripts/(publish-skill-repo.test.tsupdated + extended) — cleanbun run test:all(full pre-push gate) — clean, three times across the two review rounds, including as the actual pre-push hook on eachgit pushnodespace skill guidancereturns real seeded guidance with the provenance banner; editing a guidance node's content live and re-fetching returns the edit🤖 Generated with Claude Code