Skip to content

OpenAI-compat engine drops assistant tool_calls from replayed history, producing protocol-invalid message sequences #2198

Closed

Description

@mstomar125

Summary

OpenAiMessage serializes only role/content/tool_call_id (openai_compat_inference.rs:144-150); the ReAct loop persists the assistant turn that issued tool calls as content: "" with structured tool_calls (agent_loop.rs:1686-1691), so on the OpenAI-compat path every replayed assistant tool-call turn becomes {"role":"assistant","content":""} followed by {"role":"tool","tool_call_id":...} with no preceding tool_calls, and the tool message's name field is dropped too.

Failure scenario

On strict servers (the real OpenAI API — an explicitly supported configuration, see the gpt-4o test at line 567 — or recent vLLM): the model makes one tool call in ReAct round 1; round 2's request replays the malformed pair and the endpoint returns HTTP 400 (messages with role "tool" must be a response to a preceding message with "tool_calls"). Every tool-using conversation hard-fails immediately after the first tool execution, surfacing only an opaque engine error.

On lax servers (Ollama /v1, LM Studio): the request is accepted, but the model can never see which tool it called with which arguments — degrading multi-step tool use and defeating the duplicate-write guard's informative-result design, which assumes the model can read its own prior calls.

Location

packages/agent/src/local_agent/openai_compat_inference.rs:144-150 (the struct) and its use at :226.

This fix needs no API key — verify against local Ollama

The defect is entirely in what NodeSpace serializes, so it is verifiable without any hosted provider:

  1. Unit test on the outgoing shape (primary evidence, no network). Build a history containing an assistant tool-call turn plus its tool result, serialize, and assert:

    • tool_calls is present on the assistant message, with id, type: "function", and function.{name,arguments}
    • name is present on the tool message
    • tool_call_id still pairs the result to its call
  2. Local Ollama for the round-trip. packages/agent/tests/live_openai_compat_smoke.rs already targets http://127.0.0.1:11434/v1:

    cargo test -p nodespace-agent --test live_openai_compat_smoke -- --ignored --nocapture
    

    Ollama is enough to prove a multi-turn tool conversation still works after the change, and that nothing regressed on the path that ships today.

    ⚠️ But a green Ollama run is not evidence the bug is fixed. Ollama is one of the lax servers — it accepts the malformed shape today, which is precisely why this went unnoticed while every prior measurement ran against local Ollama. Ollama proves no regression; the unit test proves the fix.

  3. NODESPACE_PROMPT_DUMP=<path> captures the exact outgoing request JSON on this path (openai_compat_prompt_dump.rs, records tagged "engine": "openai_compat"). Reading the dumped payload is the direct way to confirm the wire shape rather than inferring it.

Strict-server confirmation is out of scope here. That belongs to #2189, which provisions a real key from a hosted OpenAI-compatible provider (AI Studio) and exercises the path end-to-end through the GUI. This issue lands first so #2189 is not blocked by a guaranteed HTTP 400 after its first tool call.

Acceptance criteria

  • OpenAiMessage carries tool_calls on assistant messages, matching the OpenAI wire contract
  • The tool message's name field is preserved
  • Unit test asserting the serialized shape for an assistant-tool-call + tool-result pair
  • live_openai_compat_smoke.rs still passes against local Ollama (no regression on the shipping path)
  • No API key or hosted provider required to verify any of the above

Severity

important — blocks #2189 and breaks every tool-using conversation against strict providers.


Found via a full-repo adversarial bug sweep, independently verified by two separate skeptical reviewers reading the actual current code before filing.

Activity

  1. self-assigned this
    on Aug 21, 2026
  2. assigned and unassigned on Aug 21, 2026
  3. added a commit that references this issue on Aug 22, 2026
    fe63b7d
  4. mstomar125 commented on Sep 3, 2026

    @mstomar125
    ContributorAuthor

    Independently re-verified before starting today's drain: PR#2236 ("Serialize assistant tool_calls and tool-result name on the OpenAI-compat path") is merged, reviewed, and additionally verified against a real Ollama tool-calling conversation. Confirmed live in openai_compat_inference.rs — OpenAiMessage now carries tool_calls. Board status was stuck on Todo despite the real GitHub closure — fixing that now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions