Audit skill retrieval text: rename description/exclusion to use_for/not_for, tighten what each skill is for, and add measured not_for entries #3522

Overview

Skills are found by semantic search over two fields. Today:

• description says what a skill is for; it is embedded with the skill's name.
• exclusion says what it is not for; it is embedded on its own and lowers the skill's score on requests that match it better than the description.

Both need an audit, and both should be renamed to say what they are:

• Rename description → use_for and exclusion → not_for. "Description" reads as prose for a person, which invites long, general text. The field is retrieval text, and authors (us, users, Playbooks) should write it short and targeted. The type definitions already describe the fields this way: "What the skill is for" / "What the skill is not for".
• Tighten each core and Playbook skill's use_for, so it targets the requests that skill should win and little else.
• Add not_for entries wherever a skill is plausibly confused with another. For example, Schema Creation is not for creating a node of a type that already exists, or for creating a saved query or view. Today only Graph Editing has one.

Problem Statement

Only one skill has an exclusion, by design. only_measured_skills_carry_an_exclusion (packages/agent/src/skill_pipeline.rs) pins the list to Graph Editing ("Remove them, delete them, get rid of them, purge them."). Its doc comment explains the rule: an exclusion that misfires only lowers the right skill's rank, and "wording that reads as equivalent was measured to behave very differently". So each exclusion must come with a live guard in packages/agent/tests/it/live_skill_retrieval_stability.rs showing that the skill's own requests are unchanged. The policy is sound; it has simply been applied to one skill, so most likely confusions have no counter-signal at all.

Descriptions are uneven. Some are long lists of verbs and examples:

• Research & Search lists eight verbs and three question forms.
• Node Deletion lists every deletion verb, and a test requires them because the description is that skill's whole retrieval signal.
• Others are one short sentence (Organization, Bulk Import).

Length and specificity vary with no stated rule. Earlier tuning (recorded in comments in skill_pipeline.rs) found that a single extra sentence can flip a top-3 result by a few thousandths.

Why an exclusion can't go in the description. An embedding has no negation, so "not for deleting" embeds near deleting (exclusion_penalized_score, packages/core/src/ops/skill_ops.rs). Confusions can only be handled by not_for, scored separately as score − λ · max(0, not_for_score − score).

Decisions

• Rename skill.description → skill.use_for and skill.exclusion → skill.not_for everywhere: core seeds, Playbook skills, user-authored skills, the typed wire type, the CLI, the frontend and docs. No alias, no fallback read of the old names; reset the database. The external SKILL.md frontmatter keeps its description key, because the Agent Skills standard requires it.

• The app shows friendly labels; the stored keys stay terse. In the skill core schema:

  Key	friendly_name (the UI label)	Field description (guidance for whoever writes it, person or agent)
  use_for	What it's for	The requests this skill should handle, worded the way someone would ask. Short and specific.
  not_for	What it's NOT for	Requests that sound similar but belong to another skill.

  "NOT" is in capitals in the label on purpose.

• use_for is short and targeted: the requests this skill should win, in the words users use. Not prose for a reader.

• not_for names what the skill is likely to be confused with: the neighbouring skills' requests that share its vocabulary.

• Every use_for and not_for change is measured, not written by intuition. The existing policy stands and is applied to every skill: a change ships only with a live retrieval guard showing that the skill's intended requests keep their rank and that the confused requests move to the right skill.

Proposed Solution

1. Rename (node type sequence)

• Follow ../nodespace-docs/development/node-type-sequence.md: rename the fields in the skill core schema and the registry.
• Set each field's friendly_name and description as in the table under Decisions. The app reads the label only through labelForField (packages/desktop-app/src/lib/utils/schema-field-label.ts).
• Decide in review whether the in-app prompt's Purpose: label for this text (render_candidates_for_prompt, packages/agent/src/local_agent/routing.rs) stays or changes. It is model-facing, so any change goes through the prompt snapshot review.
• Update SkillFields (packages/nodespace-types/src/skill.rs), then run bun run gen:types and commit the generated TypeScript.
• Update node-types.md's skill rows.
• Update every reader and writer:
  • skill_ops::find_skills and rerank_with_exclusions, and the embedding of use_for;
  • seed templates (SkillSeed, SkillFields::with_exclusion);
  • Playbook skills (packages/core/src/methodology/skills.rs);
  • the CLI (skill_description in packages/cli/src/commands/skill.rs, guidance JSON);
  • the frontend skill node;
  • SKILL.md and references/cli.md wherever they name the fields.

2. Confusion audit

• Build the confusion matrix: for each core and Playbook skill, the requests it should win and the neighbouring skills that score close on them. Use live_skill_retrieval_stability.rs and its live model, plus requests drawn from the agent eval fixtures.
• Known candidates to measure first:
  • Schema Creation vs Node Creation (create a record of an existing type) and vs query creation (save a view or filter);
  • Node Creation vs Schema Creation (a kind of thing not tracked yet);
  • Graph Editing vs Node Deletion (already guarded) and vs Relationship Management (point a record at another);
  • Research & Search vs Graph Editing on lookups that name a state;
  • Relationship Management vs Organization (link vs add to a collection);
  • Node Merge vs Conflict Journal (combine duplicates vs inspect collisions);
  • Bulk Import vs Node Creation (one record vs a document);
  • Playbook skills (Creating an Issue, Sprints and Cycles, Issue Validation Rules) vs the core skills they overlap.

3. Tighten and add, measured

• Rewrite each use_for toward the short, targeted form. Keep a term only where removing it measurably loses a request; the Node Deletion verb test is an example.
• Add a not_for wherever the audit shows a confusion that use_for wording alone can't fix.
• Replace only_measured_skills_carry_an_exclusion's fixed list with a test that every skill with a not_for has a named live guard. The rule stays deliberate; its single-entry list goes.

Architecture Impact

• ADR-064 rule 3 (../nodespace-docs/decisions/064-instruction-channel-doctrine.md, "Skill descriptions own retrieval matching") stands; it is reworded for the new field names. This issue applies the rule more strictly, it doesn't reverse it.
• ADR-086 (core node types defined once): the rename follows the node type sequence.
• Companion nodespace-docs changes in the same change:
  • components/node-types.md (skill rows);
  • architecture/prompt-skill-system.md (the skill field list and retrieval description);
  • ADR-064 rule 3's wording;
  • components/creating-node-types.md or any authoring guide that tells users how to write a skill's description.

Acceptance Criteria

• skill.description and skill.exclusion are renamed to skill.use_for and skill.not_for in the schema, registry, wire type, generated TypeScript, CLI output, frontend and docs; nothing reads the old names
• Core and Playbook seeds use the new names
• bun run node-types:check and bun run types:check pass
• The app labels the two fields "What it's for" and "What it's NOT for", and each field's schema description carries the authoring guidance from the Decisions table
• Every core and Playbook skill's use_for has been reviewed against a recorded confusion matrix; the matrix (requests, expected skill, scores before and after) is kept with the live suite
• Every core skill has a not_for where the matrix shows a confusion its use_for can't fix — at least Schema Creation (records of an existing type; saved queries and views), and every other confusion the matrix finds
• Each not_for and each changed use_for has a live guard in live_skill_retrieval_stability.rs. The guards show that the skill's intended requests keep their rank, and that the confused requests rank the right skill first
• No existing live guard regresses
• only_measured_skills_carry_an_exclusion is replaced by a test that every skill with not_for names its live guard
• Prompt and routing golden snapshots regenerated with UPDATE_GOLDEN=1 and the diff reviewed
• Docs updated as listed under Architecture Impact
• Code passes bun run quality:fix
• Tests pass (bun run test:changed), and the live retrieval suite passes on the locked model

Technical Specifications

Reference Files

• packages/nodespace-types/src/skill.rs — SkillFields.description, .exclusion
• packages/agent/src/skill_pipeline.rs — SKILL_SEEDS (descriptions, exclusions, measurement comments), only_measured_skills_carry_an_exclusion, the Node Deletion verb test
• packages/core/src/ops/skill_ops.rs — exclusion_penalized_score, rerank_with_exclusions, find_skills
• packages/core/src/methodology/skills.rs — Playbook skill templates
• packages/agent/tests/it/live_skill_retrieval_stability.rs — the live guards
• packages/cli/src/commands/skill.rs — skill_description, guidance JSON
• ../nodespace-docs/development/node-type-sequence.md — the sequence the rename follows

Implementation Details

• The live suite needs the embedding model. Measure on the locked model, and record the scores the decisions rest on, as the existing comments do.
• Changing a seed's use_for or not_for changes its embedding; reset the database rather than re-embedding existing rows.
• Model-facing text in packages/agent changes, so prompt_assembly_snapshot needs regenerating.

Related Issues

• Let agents fetch NodeSpace's skills and schemas by intent: make skill guidance return them, and embed schemas #3502 (external agents fetch skills; ranking there uses the same fields)
• Keep Playbooks out of the agent's vocabulary: agents work Plays and schemas, and Playbooks install only from the app #3506 (Playbook skill wording)

Non-Goals

• Changing the exclusion scoring formula or λ
• Changing routing bars