Skip to content

Refine local agent system prompt for better tool usage and response quality #1040

Closed

Description

@malibio

The local agent system prompt in prompt_templates.rs is a placeholder. It needs refinement based on real-world usage patterns with the Ministral 3B/8B models.

Current State

  • Basic system prompt with node reference URI format and display formatting instructions
  • Model sometimes makes bad tool calls (nesting args under properties, echoing schema)
  • Model sometimes retries tool calls unnecessarily instead of responding with results
  • Tool definitions are duplicated between MCP handlers and local agent GraphToolExecutor

Areas to Refine

Prompt Quality

  • Tool usage guidance: when to call tools vs answer directly
  • Response formatting: how to present search results, node references
  • Error handling: what to do when tool calls return empty results
  • Conversation flow: when to ask clarifying questions vs make assumptions
  • Token budget awareness: keep prompt concise for 3B model context limits

Tool Unification

  • MCP handlers (nodespace-core) and local agent tools (GraphToolExecutor) have separate implementations of the same operations
  • get_node markdown export now calls MCP handler directly — same pattern should apply to all tools
  • Refactor shared tool logic into a common service layer that both MCP handlers and GraphToolExecutor use
  • Ensures consistent behavior, descriptions, and parameter schemas across both surfaces

Acceptance Criteria

  • System prompt refined based on observed model behavior
  • Tool call accuracy improved (fewer bad/redundant calls)
  • Response quality improved for common queries (search, create, update)
  • Prompt fits within 4k token budget for system prompt + tool definitions
  • Shared tool logic extracted — MCP handlers and GraphToolExecutor use the same underlying implementation
  • Tool descriptions and parameter schemas are consistent between MCP and local agent

Activity

  1. malibio commented on Apr 5, 2026

    @malibio
    CollaboratorAuthor

    ADR-030: Prompt & Skill Node Architecture

    A follow-up architecture has been designed (ADR-030 in nodespace-docs/decisions/030-prompt-skill-node-architecture.md) that builds on top of this issue's work:

    • The hardcoded prompts refined here become the permanent base layer that graph-stored prompt nodes override
    • The ops/ layer created here becomes the shared backend that skill nodes scope via tool_whitelist
    • The context_ops.rs created here gets reused by the PromptAssembler for template variable injection
    • The packages/agent/ crate created here is where the PromptAssembler and skill injection pipeline will live

    No changes needed to this issue's plan. ADR-030 phases will be separate follow-up issues that depend on #1040 being complete.

    Key design decisions from ADR-030 relevant to this work:

    • Prompts-as-nodes override hardcoded defaults, never replace them
    • Skills wrap tools with rich guidance and are discovered via semantic search at the system layer (not by the model)
    • Intent extraction uses pattern matching (no extra inference call)
    • One active skill per turn, tool whitelist scopes available tools
  2. self-assigned this
    on Apr 5, 2026
  3. malibio commented on Apr 5, 2026

    @malibio
    CollaboratorAuthor

    Baseline: Frontend 128 files, 3918 tests all passed. Starting implementation — ops layer + agent crate relocation + domain-aware prompt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions