Skip to content

create_relationship's tool description recommends a relation name the validator rejects #2234

Closed

Description

@malibio

Summary

create_relationship's tool description tells the model to fall back to a generic relation label, and names related_to as an example. The validator refuses it. The model follows its instructions exactly and the write is rejected.

Evidence

Surfaced by the first live run of the re-themed agent matrix (#1977, PR #2232). The model recovered both node ids correctly and picked the direction correctly:

create_relationship is_error=true duration_ms=1
args={"from_id":"1fb911ba-49dd-44ae-846b-aad0766beaee",
      "relationship_type":"related_to",
      "to_id":"c26d6108-605a-425f-b12e-6a0dd5bd3e2a"}
result={"error":"tool execution failed: create_relationship failed: Validation failed:
        Relationship 'related_to' not defined in schema 'task'.
        Built-in relationships (member_of, has_child, mentions, has_role) are universal."}

The contradiction

packages/agent/src/local_agent/tools.rs:1055 — relationship_type's description:

"The relation's name, lowercase snake_case. Use a name defined on the relevant schema(s) (e.g. 'supersedes', 'has_task') if one applies, otherwise a generic label (member_of, mentions, related_to, etc.)."

packages/core/src/services/node_service/relationship.rs:548 — the validator:

"Relationship '{}' not defined in schema '{}'. Built-in relationships (member_of, has_child, mentions, has_role) are universal."

related_to is offered as a generic fallback and is not in the universal set. Two of the three examples the description gives as generic labels (member_of, mentions) are legal; the third is not.

Impact

An ad-hoc link between two nodes whose schemas define no relationships cannot be recorded at all — e.g. a task and a text node. This is the common case for the product's own framing ("a decision means nothing without the work it constrains"): connecting a piece of work to the decision that governs it requires either a schema that predefines the relation or falling back to mentions, whose name does not carry the intended meaning.

The failure is loud (the model reports "relationship creation failed") rather than silent, which is the good half. The bad half is that the guidance channel actively steers toward the rejected value.

Options

  1. Narrow the description to what the validator accepts — cheapest, and consistent with ADR-064 rule 1 (argument constraints belong on the tool schema). Could go further and declare relationship_type as an enum of the universal set plus the source schema's own relations, which makes the constraint structural instead of prose.
  2. Add related_to to the universal set — matches what the description already promises, but adds a semantically empty edge type to every node.
  3. Accept any relation name between nodes whose schemas define none — largest change, and weakens the guarantee the validator exists to provide.

Option 1 looks right; the choice between "trim the prose" and "declare the enum" is worth deciding deliberately, since the enum version is the one that measurably works (ADR-064 rule 1).

Not in scope of the eval PR

PR #2232 works around this by having the scenario ask for a link that maps onto mentions, so the eval stays winnable. That is a fixture-side accommodation, not a fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions