Skip to content

create_node is refused when the model sends a type's display name in place of its id #3666

Closed

Description

@malibio

Overview

Asked to add a record of a type the user created a turn earlier, the local agent calls create_node with the type's display name as node_type ("feature spec"), where the type's id is feature_spec. The call is refused as an unknown type and no record is created. The routing eval's instance-not-schema-invoice scenario fails on it.

Measured on gemma-4-e4b-q4km, isolated daemon.

Problem Statement

instance-not-schema-invoice runs "Create a database for tracking our feature specs" as a prior turn, then scores "Add a spec for offline sync that's due for sign-off next Friday".

The scored turn's call, the same in each failing rep:

create_node {"content":"Offline Sync Spec","field_values":{"due_date":"2026-10-13"},"node_type":"feature spec"}
→ Unknown node_type 'feature spec': no such core type or schema. Create the schema first with create_schema, or use an existing type id.

In one rep the turn first called create_schema for feature spec again, was told the type exists as feature_spec, and still sent "feature spec" to create_node.

Results:

The scenario only became visible as a record that was never created once the routing eval scored record creation on an accepted create_node call (createsRecord, added in #3655). Before that it passed or failed on which tools were called.

create_node's node_type description already tells the model to copy the id exactly from the EXISTING SCHEMAS block, "character for character, including underscores". The block renders the type as - feature_spec "feature spec" -> status: text, with the id first and the display name quoted after it.

Proposed Solution

None is known. Candidates, each to be measured on the scenario at three reps before it is chosen:

  • Resolve a node_type that names no type but is the display name of exactly one, at the local agent's tool boundary, and say so on the result. ADR-064's amendments record this shape working twice where a more informative refusal did not: the tool does what its description asks.
  • Change how the EXISTING SCHEMAS line renders the id and the display name, so the name is not the more copyable of the two.
  • Derive schema ids in kebab-case so every type id follows the naming convention #3646 changes how type ids are derived (kebab-case). Whether a display name is then closer to or further from its id is worth measuring with it, not separately.

Documentation

  • ADR-064 rule 1 and rule 4, and the 2026-10-05 amendments - tool schemas own argument shape; a refusal is a result the model has to turn into a second call, and what was done in two cases where it could not
  • ADR-038 - the offered set of types a turn is held to, which a resolved name must still respect
  • Agent evals - the routing layer, createsRecord, and how to run part of a fixture
  • Updates: ADR-064, if the tool boundary resolves a display name

Architecture Impact

No ADR is reversed. A tool-boundary resolution would be a third instance of the pattern ADR-064's 2026-10-05 amendments record, and belongs there with its measurement.

Acceptance Criteria

  • instance-not-schema-invoice creates the record in 3 of 3 reps
  • The call that creates it names the type the prior turn created, and no second type is created
  • A node_type that matches no type and no single display name is still refused
  • A turn held to an offered set of types cannot reach a type outside it through a display name
  • Code passes bun run quality:fix and bun run test:changed

Technical Specifications

Reference Files

  • create_node definition and execution: packages/agent/src/local_agent/tools.rs - def_create_node, exec_create_node, off_menu_type
  • The refusal: packages/core/src - search for "Unknown node_type"
  • How the schemas block renders a type: packages/core/src/ops/entity_types_block.rs - EntityTypeDescriptor::render_line
  • Id from a name: packages/core/src/services/node_service/mod.rs - normalize_schema_id
  • The scenario: scripts/eval/fixtures/routing.ts - instance-not-schema-invoice

Reproducing

Isolate the daemon with NODESPACE_HOME and NODESPACED_MODEL_PATH (see "Setup" in agent-eval.md), load gemma-4-e4b-q4km, then:

bun run eval:routing e4b out.json --runs 3 --between-runs <reset> --only instance-not-schema-invoice

Tool arguments and error text are in the daemon log's Tool executed lines at RUST_LOG=debug.

Related Issues

Activity

  1. malibio commented on Oct 7, 2026

    @malibio
    CollaboratorAuthor

    Frontend: 5863 passed (323 files)

  2. self-assigned this
    on Oct 7, 2026
  3. added a commit that references this issue on Oct 7, 2026
    c8462ac
  4. malibio commented on Oct 7, 2026

    @malibio
    CollaboratorAuthor

    The scenario id instance-not-schema-invoice is renamed to instance-not-schema-spec (PR #3688); the scenario is about feature specs. Use the new id in --only when reproducing. PR #3685 (display-name resolution) is merged; the 3-rep run and the ADR-064 amendment are still open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions