Non-buffered Chat Completions tool-call streaming finalizes incomplete tool calls with empty name/call_id

Please read this first

Have you read the docs? Agents SDK docs
Have you searched for related issues? Searched for Chat Completions streaming / buffer_streamed_tool_calls / incomplete tool-call handling; did not find an existing report of this default-vs-buffered asymmetry.

Describe the bug

For OpenAI Chat Completions streaming tool calls, the default path (buffer_streamed_tool_calls=False) and the buffered path (buffer_streamed_tool_calls=True) disagree on incomplete tool-call streams:

Path	Incomplete tool call (missing `id` and/or function `name`)
Default / non-buffered	Still finalizes a `ResponseFunctionToolCall` with empty `name` and/or `call_id` via the end-of-stream fallback
Buffered	Raises `ModelBehaviorError` (`without a tool call id` / `without a function name`)

The same incomplete upstream stream can either hard-fail or silently produce an unusable (or partially usable) tool call, depending only on a flag that defaults to False. LiteLLM/AnyLLM integrations always use the non-buffered path today.

This appears to come from ChatCmplStreamHandler.handle_stream:

On first tool-call delta for an index, it seeds ResponseFunctionToolCall(name="", call_id="", ...).
Live streaming only starts once both name and call_id are present.
At stream end, if streaming never started, the "fallback to old behavior" branch still emits response.output_item.added / arguments delta / response.output_item.done with those empty fields.

The buffered path validates completeness in _buffered_tool_call_delta and raises instead (added in PR #3506).

Debug information

Agents SDK version: 0.18.3 (checkout 965335aba6f6c71500e0b8cdb4e9e495f5801d4d)
Python version: Python 3.12.10
OS: Windows 11
API surface: Chat Completions streaming (OpenAIChatCompletionsModel / OpenAIProvider(use_responses=False))
Flag: buffer_streamed_tool_calls default False vs True
Reproduction count: 2/2 stable (unit-level synthetic stream; no live API key)

Repro steps

Minimal unit-style repro against the stream handler (no live API key required):

import asyncio
from typing import AsyncIterator

from openai.types.chat.chat_completion_chunk import (
    ChatCompletionChunk,
    Choice,
    ChoiceDelta,
    ChoiceDeltaToolCall,
    ChoiceDeltaToolCallFunction,
)
from openai.types.responses import Response

from agents.models.chatcmpl_stream_handler import ChatCmplStreamHandler
from agents.exceptions import ModelBehaviorError


def _chunk(tool_calls=None, finish_reason=None) -> ChatCompletionChunk:
    return ChatCompletionChunk(
        id="chunk-id",
        created=1,
        model="fake",
        object="chat.completion.chunk",
        choices=[
            Choice(
                index=0,
                delta=ChoiceDelta(tool_calls=tool_calls),
                finish_reason=finish_reason,
            )
        ],
    )


async def incomplete_stream() -> AsyncIterator[ChatCompletionChunk]:
    # Incomplete: index + partial args, but no tool call id and no function name.
    yield _chunk(
        tool_calls=[
            ChoiceDeltaToolCall(
                index=0,
                function=ChoiceDeltaToolCallFunction(arguments='{"a":'),
            )
        ]
    )
    yield _chunk(finish_reason="tool_calls")


async def main() -> None:
    response = Response(
        id="resp",
        created_at=1,
        model="fake",
        object="response",
        output=[],
        parallel_tool_calls=False,
        tool_choice="auto",
        tools=[],
    )

    # Default / non-buffered
    events = [
        event
        async for event in ChatCmplStreamHandler.handle_stream(
            response, incomplete_stream()
        )
    ]
    done_items = [
        e.item
        for e in events
        if getattr(e, "type", None) == "response.output_item.done"
    ]
    print("non-buffered done items:", done_items)
    # Observed: function_call with name="" and call_id=""

    # Buffered
    try:
        async for _ in ChatCmplStreamHandler.handle_stream(
            response,
            ChatCmplStreamHandler.buffer_tool_call_stream(incomplete_stream()),
        ):
            pass
        print("buffered: no error")
    except ModelBehaviorError as exc:
        print("buffered raised:", type(exc).__name__, str(exc))
        # Observed: ModelBehaviorError about missing tool call id / function name


asyncio.run(main())

Related existing coverage asserts the buffered hard-fail only:

tests/models/test_openai_chatcompletions_stream.py::test_buffered_tool_call_delta_requires_id_and_name
tests/models/test_openai_chatcompletions_stream.py::test_stream_response_buffered_tool_calls_raise_for_missing_tool_call_delta

I did not find a matching assertion that the default path rejects the same incomplete stream.

Expected behavior

Buffered and non-buffered Chat Completions streaming paths should treat incomplete tool-call streams consistently.

If a tool call never receives a name and/or call_id, the default non-buffered path should fail closed the same way as the buffered path (ModelBehaviorError), rather than emitting a finalized function call with empty identifiers.

(If empty finalization is intentionally kept for backward compatibility, that should be documented, and ideally warned/gated rather than silent.)

Actual behavior

Non-buffered (default): end-of-stream fallback finalizes incomplete tool calls with empty name / call_id.
Buffered: raises ModelBehaviorError when id or name is missing.

Additional observation from runner path analysis: empty name tends to fail later as "tool not found"; a present name with empty call_id can still reach tool execution with an empty call_id, which may break correlation/history pairing.

Possible cause

This may be legacy fallback behavior in ChatCmplStreamHandler.handle_stream (comment about "function name never arrived"), while buffer_tool_call_stream later added strict completeness checks in PR #3506. The safer checks are behind a non-default flag, so default and LiteLLM remain on the permissive finalizer.

Relevant spots:

src/agents/models/chatcmpl_stream_handler.py — empty seed; stream gate; end-of-stream fallback
src/agents/models/chatcmpl_stream_handler.py — _buffered_tool_call_delta validation
src/agents/models/openai_chatcompletions.py — buffer_streamed_tool_calls: bool = False

Suggested direction

Align non-buffered finalization with buffered validation (raise ModelBehaviorError when name or call_id is still empty at stream end), or
If empty finalization must remain for compatibility, document it and consider a warning / opt-in strict mode; prefer fail-closed by default for incomplete identifiers.
Add a regression test that feeds incomplete tool-call deltas through both paths and asserts matching failure (or explicit drop) behavior.
Optional defense-in-depth: reject empty call_id before tool execution in the runner.

I'd be happy to prepare a PR with a regression test if this behavior is confirmed.

Additional context

Related prior art (not a duplicate bug report): PR feat: add buffered Chat Completions tool-call streaming #3506
Nearby but different: chat completion model failed to produce multiple tool call items #989, Chat Completions streaming fallback tool calls can reuse output_index #3104 / fix: #3104 stabilize chat completions tool call output indexes #3161 (stream/tool indexing)

Could you provide a redacted raw stream from a real provider that ends without the ID or function name, together with the provider, model, adapter (OpenAIChatCompletionsModel, LiteLLM, or AnyLLM), versions, and observed downstream failure? For OpenAIProvider, please also confirm whether buffer_streamed_tool_calls=True avoids the problem. That evidence will determine whether a fix belongs in a specific adapter or the shared finalization path.

Thanks for the review — here is the additional evidence you asked for.

What we can share today

We reproduced this deterministically without a live OpenAI key by feeding incomplete Chat Completions chunks into the same conversion layer production streaming uses: ChatCmplStreamHandler.handle_stream (also used by LiteLLM/AnyLLM, which never enable buffering).

Environment

Agents SDK: 0.18.3 @ 965335aba6f6c71500e0b8cdb4e9e495f5801d4d
openai Python package: 2.46.0
OS: Windows 11 / Python 3.12.10
Adapter path: Chat Completions stream handler (shared by OpenAIChatCompletionsModel, LiteLLM, AnyLLM)

Incomplete stream shape (synthetic, redacted of any secrets)

Equivalent chunk sequence (arguments-only tool_call delta, then finish_reason=tool_calls, no id / function name):

data: {"id":"chunk-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"type":"function","function":{"arguments":"{\"a\":"}}]},"finish_reason":null}]}

data: {"id":"chunk-2","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}

data: [DONE]

(Real providers that interleave id/name on later packets, or cut the TCP/SSE stream after argument deltas, produce the same incomplete state at end-of-stream.)

Results (2/2 stable)

Mode	Result
Default non-buffered (`buffer_streamed_tool_calls=False`)	Finalizes `ResponseFunctionToolCall(name="", call_id="")` via end-of-stream fallback; no `ModelBehaviorError`
Buffered (`buffer_tool_call_stream` / `buffer_streamed_tool_calls=True`)	`ModelBehaviorError`: Buffered Chat Completions tool call stream ended without a tool call id.

So for OpenAIProvider with buffer_streamed_tool_calls=True: yes — that path avoids silent empty finalization and fails closed, matching what you asked to confirm.

Downstream impact observed in code path analysis

Empty name → later dispatch typically surfaces as tool-not-found (Tool not found…).
Present name + empty call_id can still reach tool execution with call_id="" (correlation/history risk).
LiteLLM/AnyLLM always use the non-buffered handler today, so they stay on the permissive finalization path.

Why this still seems like the shared finalization path

Buffered validation lives in _buffered_tool_call_delta (PR #3506), while the default finalizer still emits incomplete items. A real flaky/truncated provider stream would hit the same shared fallback regardless of adapter, unless that adapter opts into buffering.

I do not currently have a production customer capture of a third-party gateway dropping id/name mid-stream. If you prefer to gate a code change on a live provider capture, I can try to obtain one next; otherwise I’m happy to open a PR that:

fail-closes incomplete name/call_id on the non-buffered finalizer (parity with buffered), and
adds a unit regression on synthetic incomplete chunks (no network).

Happy to adjust the PR scope based on your preference (shared finalizer vs adapter-only).

Thanks for the additional details. This confirms the synthetic handler behavior and that buffer_streamed_tool_calls=True fails closed, but it does not yet establish the real provider path we asked about. The stream is still manually constructed, and the statements about providers reordering fields or a truncated connection ending in the same state are not backed by a named provider capture or transport-level reproduction.

Please hold off on opening a PR until you can provide either a redacted raw stream from a named provider or a reproduction showing that a real transport stream ends cleanly without the ID/name, together with the observed runner outcome. That evidence is necessary because the current fallback is released compatibility behavior. Once we have it, we can decide whether the fix belongs in a specific adapter or the shared finalizer.

Navigation Menu

Repository navigation

Non-buffered Chat Completions tool-call streaming finalizes incomplete tool calls with empty name/call_id #3861

Please read this first

Describe the bug

Debug information

Repro steps

Expected behavior

Actual behavior

Possible cause

Suggested direction

Additional context

What we can share today

Incomplete stream shape (synthetic, redacted of any secrets)

Results (2/2 stable)

Downstream impact observed in code path analysis

Why this still seems like the shared finalization path

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Non-buffered Chat Completions tool-call streaming finalizes incomplete tool calls with empty name/call_id #3861

Description

Please read this first

Describe the bug

Debug information

Repro steps

Expected behavior

Actual behavior

Possible cause

Suggested direction

Additional context

Activity

seratch commented on Jul 17, 2026

baiyuxi930826 commented on Jul 17, 2026

What we can share today

Incomplete stream shape (synthetic, redacted of any secrets)

Results (2/2 stable)

Downstream impact observed in code path analysis

Why this still seems like the shared finalization path

seratch commented on Jul 17, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions