Skip to content

Add fetch-at-activation guidance mechanism for external agent skill sessions - #2559

Merged
mstomar125 merged 2 commits into
mainfrom
issue-2532-skill-fetch-guidance
Sep 10, 2026
Merged

mstomar125 merged 2 commits into
mainfrom
issue-2532-skill-fetch-guidance

Conversation

@mstomar125

@mstomar125 mstomar125 commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Closes the two genuinely-unbuilt pieces #2532 tracked, implementing #1958's fetched, hybrid-shaped delivery-channel decision:

  • Fetch-at-activation mechanism: nodespace skill guidance <query> (packages/cli) fetches procedural guidance from the graph's seeded skill nodes via the existing NodeService.SearchNodes RPC (node_type=skill, subtree markdown attached) — the same content the in-app agent already retrieves via semantic routing, now reachable by an external harness through the CLI. No new server/proto surface was needed.
  • Provenance, not silent merging: every result is wrapped in a marker — a GRAPH-FETCHED GUIDANCE banner in human mode, a "provenance": "graph-fetched" envelope in --json — so fetched content can never be indistinguishable from the skill's own static instructions. SKILL.md's body gained a ## Graph-Authored Guidance (Fetched) section instructing an activated agent to run this command before a nontrivial operation, and a new on-demand reference (references/graph-authored-guidance.md) covers the trust boundary in depth: fetched content can supply procedure, never permission, and any fetched instruction that tries to skip a confirmation or claims system/developer authority is itself the signal of a compromised graph entry, not something to act on.
  • Wired through every place that ships packages/skill's reference tier, not just the body: all five installer targets' shims lists (agents.ts), the PTY-spawned-agent path's embedded reference tier (context_assembly.rs — a mechanical 1:1 addition mirroring the two existing embeds, required because that path reuses the identical SKILL.md/references/ verbatim), and the public-repo publish script (publish-skill-repo.ts — already derives its file set dynamically from agents.ts's shim lists, so no logic change was needed there, only stale comments/test expectations).

The seeding-skips-by-node-type prerequisite is already fixed

Investigated before designing anything: the prerequisite #2532 listed as still-needed ("seeding skips by node type, so guidance changes never reach an existing install") was already fixed by PR #1818 (closed #1812, merged 2026-07-30 — per-node versioned reconciliation replacing the type-level skip), predating both #1958 and #2532. New tests (packages/agent/tests/skill_guidance_fetch_mechanism.rs) confirm this holds against the real seed_skill_nodes() registry, not just the synthetic type PR #1818's own tests covered:

Both were also verified live end-to-end: real isolated daemon (NODESPACE_HOME-scoped), real embedding model, real nodespace skill guidance CLI call, real node update on a seeded guidance node — the edit showed up in the next fetch.

What's honestly still open

The manual harness matrix against Codex (and ideally Gemini CLI/Cursor) is not attempted here — it needs a machine with those CLIs installed, which this environment doesn't have. Only the Claude Code leg was previously verified (on #1958, 2026-08-25). This remains the one open acceptance criterion on #2532.

Design calls made (documented, not silently assumed)

Review feedback addressed (round 1)

A deep adversarial review found three real issues, all fixed:

  1. --limit/include_markdown mismatch (real bug): --limit above 5 was sent uncapped to the request while only include_markdown was clamped, so results past the 5th printed a full provenance banner around empty content. Fixed by capping limit itself in a new, unit-tested build_guidance_request — a guidance entry's whole value is its markdown content, so there's no reason to ever request more than the server attaches markdown to.
  2. 8 bare issue/PR-number references in code comments (skill.rs, skill_guidance_fetch_mechanism.rs) violated CLAUDE.md's "describe the behavior/constraint directly" rule despite evading the automated checker's regex. All reworded to describe the behavior or history directly instead of citing numbers.
  3. Provenance-banner spoofing hardening: print_guidance's human-mode banners wrapped fetched content verbatim and unescaped, with no defense against a malicious graph node forging a banner-shaped line (fake closing delimiter, fake second banner) or hiding the real banner with raw ANSI escapes. Fixed two ways: (a) every banner now carries a per-invocation random tag (fresh UUIDv4, generated at call time and announced once at the top of the output) that content authored before the call cannot have predicted, so a banner-shaped line without the announced tag is recognizable as content, not a boundary; (b) fetched title/description/markdown content is passed through a new sanitize_for_terminal that strips ANSI CSI sequences and other control characters before printing, so raw escape bytes can no longer redraw or conceal the banner. --json output is untouched — it was already confirmed unspoofable by construction (a JSON string value can't be mistaken for a structural delimiter).

New/changed tests for all three were mutation-tested individually (the underlying fix reverted, confirmed the test then fails, restored). Full gate (bun run test:all) re-run clean, bun run quality:fix re-run clean.

Also noted, not a blocker: SKILL.md's body is currently ~4957 tokens (313 lines) — under the 5000-token/500-line spec recommendation the skill_md_body_is_within_spec_size_recommendations test guards, but with only ~43 tokens of headroom. Flagging so it's on record before the next addition to the body.

Test plan

  • cargo test -p nodespace-cli -p nodespace-agent — all green, including new provenance/formatting/hardening unit tests (mutation-tested individually) and the new data-layer reconciliation/edit-visibility tests
  • bun run --cwd packages/skill test — 91/91 pass
  • bun run skill:check / skill_md_generation.rs (11/11, including the token/line budget guard) — clean, no drift
  • bun test scripts/ (publish-skill-repo.test.ts updated + extended) — clean
  • bun run test:all (full pre-push gate) — clean, three times across the two review rounds, including as the actual pre-push hook on each git push
  • Live smoke test against a real isolated daemon + real embedding model: nodespace skill guidance returns real seeded guidance with the provenance banner; editing a guidance node's content live and re-fetching returns the edit

🤖 Generated with Claude Code

…essions

Implements the two unbuilt pieces #2532 tracked for issue #1958's fetched,
hybrid-shaped delivery-channel decision:

- `nodespace skill guidance <query>` (packages/cli) fetches procedural
  guidance from the graph's seeded `skill` nodes via the existing
  NodeService.SearchNodes RPC (node_type=skill, subtree markdown attached),
  wrapping every result in a provenance marker -- a `GRAPH-FETCHED
  GUIDANCE` banner in human mode, a `"provenance": "graph-fetched"`
  envelope in `--json` -- so fetched content is never silently
  indistinguishable from the skill's own static instructions.
- SKILL.md's body gains a `## Graph-Authored Guidance (Fetched)` section
  instructing an activated agent to run this command before a nontrivial
  operation, and a new on-demand reference
  (references/graph-authored-guidance.md) spells out the trust boundary:
  fetched content can supply procedure, never permission, and any fetched
  instruction that tries to skip a confirmation or claims system/developer
  authority is itself the signal of a compromised graph entry.
- The new reference file is wired through every place that ships
  packages/skill's reference tier: all five installer targets'
  `shims` lists (agents.ts), the PTY-spawned-agent path's embedded
  reference tier (context_assembly.rs, mechanical addition mirroring the
  two existing embeds -- required because that path reuses the same
  SKILL.md/references/ verbatim), and the public-repo publish script
  (publish-skill-repo.ts, already derived dynamically from agents.ts's
  shim lists so no logic change was needed there, only comments/tests).

Also verifies, via new tests against the real seed_skill_nodes() registry,
that the seeding-skips-by-node-type prerequisite #2532 listed as
still-needed is already fixed (PR #1818/#1812, per-node versioned
reconciliation, merged 2026-07-30 -- predating both #1958 and #2532): a
compiled-template content change reaches an already-seeded database on
next seed, and a live graph edit is immediately visible to a fetch with no
reseed or restart. Manually verified end-to-end against a real isolated
daemon + embedding model: edited a seeded guidance node's content, then
confirmed `nodespace skill guidance` returned the edit.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…banner against spoofing

- Cap `--limit` itself (not just include_markdown) at the server's
  markdown-attachment ceiling in a new build_guidance_request(), so a
  guidance entry can no longer print a full provenance banner around empty
  content when more than 5 results are requested.
- Reword code/test comments that cited issue or PR numbers to describe the
  underlying behavior or history directly instead.
- Harden print_guidance's human-mode provenance banner against a malicious
  or corrupted graph node forging a banner-shaped line: each banner now
  carries a per-invocation random tag (fresh UUIDv4, generated at call
  time and announced once at the top of the output) that fetched content
  authored before the call cannot have predicted. Fetched title,
  description, and markdown content is also passed through a new
  sanitize_for_terminal that strips ANSI CSI sequences and other control
  characters, so raw escape bytes in fetched content can no longer redraw
  or conceal the banner. --json output is unchanged (already unspoofable
  by construction).

Also updates the reference doc to explain the tag mechanism to an agent
reading it, and adds a uuid dependency to nodespace-cli for the tag.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mstomar125

Copy link
Copy Markdown
Contributor Author

All three review findings addressed in 93f034c1:

  1. --limit/include_markdown mismatch — fixed by capping limit itself (not just include_markdown) in a new, unit-tested build_guidance_request(). Chose "cap it" over "fix the doc comment": a guidance entry with no markdown is useless, so there's never a reason to request more than the server attaches markdown to.
  2. 8 bare issue/PR references in code comments — all reworded to describe the behavior/history directly.
  3. Provenance-banner spoofing hardening — every human-mode banner now carries a per-invocation random tag (fresh UUIDv4, announced once at the top of the output) that pre-authored fetched content cannot predict, plus a new sanitize_for_terminal strips ANSI/control characters from fetched title/description/markdown before printing. --json untouched (already confirmed unspoofable).

Also done (non-blocking items): filed #2560 for the SeedTier::System durability tension and referenced it in the PR description; noted SKILL.md's current token count (~4957/5000) in the description.

Every new/changed test was mutation-tested individually (reverted the fix, confirmed the test fails, restored) — including one real miss caught in the process: the first version of the tag-embedding test used a substring check that passed even when only the closing banner carried the tag, because the closing banner's text contains the opening banner's check string as a substring. Fixed the assertion to anchor on "=== GRAPH-FETCHED" (which the closing "=== END GRAPH-FETCHED..." line doesn't contain), then re-confirmed the mutation is caught.

Full gate (bun run test:all) clean, quality:fix clean, both re-run as the actual pre-push hook on this push.

@mstomar125

Copy link
Copy Markdown
Contributor Author

Re-reviewed (focused pass on the 3 fixes, isolated against commit 93f034c): the limit fix confirmed to cap the actual SearchRequest.limit, not just include_markdown, with a direct unit test at the exact point of the original bug. All 8 bare issue-number references confirmed gone (grepped the full diff), reworded text preserves the original meaning. The provenance-hardening fix — the real crux here — confirmed sound: a fresh UUIDv4 per invocation embedded in both banners (unpredictable to static fetched content), sanitize_for_terminal() comprehensively strips CSI/control sequences from every fetched field. The mutation-test fix is genuine — manually confirmed the final test's anchor string is no longer a substring of the closing banner, closing the false-positive gap. #2560 is real and well-scoped. Net diff (+319/-59) matches exactly, no scope creep. Merging.

@mstomar125
mstomar125 merged commit ac9d846 into main Sep 10, 2026
@mstomar125
mstomar125 deleted the issue-2532-skill-fetch-guidance branch September 10, 2026 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Seeded content never reaches an existing install: replace type-level skip with per-node versioned re-seed

1 participant