Skip to content

fix(streaming): announce the chat completions assistant message once - #4275

Merged
seratch merged 1 commit into
openai:mainfrom
hsusul:fix/chatcmpl-stream-duplicate-message-added
Aug 7, 2026
Merged

seratch merged 1 commit into
openai:mainfrom
hsusul:fix/chatcmpl-stream-duplicate-message-added

Conversation

@hsusul

@hsusul hsusul commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Component: src/agents/models/chatcmpl_stream_handler.py — ChatCmplStreamHandler.handle_stream, the assistant-message content-part branches.

Problem. A Chat Completions stream whose assistant message carries both a text part and a refusal part emits two response.output_item.added events for that single message, and only one response.output_item.done. Both content-part openers (delta.content and delta.refusal) unconditionally construct a fresh in-progress ResponseOutputMessage and announce it, even when the sibling part already opened that same message at the same output_index.

The both-parts case is not hypothetical: the handler explicitly supports it — each branch offsets its content_index by the sibling part (content_index += 1 / refusal_index += 1), and #4236 just landed to make the completed response respect those indexes. Non-OpenAI providers routed through this handler (LiteLLM, any-llm, and other OpenAI-compatible chat backends, which all share ChatCmplStreamHandler) are where a content delta and a refusal delta arrive in the same turn.

Impact. Consumers of the raw streamed events (raw_response_event) that track open output items keyed by output_index see an item opened twice and closed once. Reconstructing the response from the raw stream yields a duplicate assistant message, or leaves one item permanently open. The Responses API — which this handler emulates — emits exactly one response.output_item.added per output item. RunResult is unaffected, because it reads the final response.completed payload.

Repro (before this change):

events = await ChatCmplStreamHandler.handle_stream(response, stream)  # refusal chunk, then content chunk
response.created
response.output_item.added      output_index=0 item=message   <-- 1st
response.content_part.added     content_index=0 part=refusal
response.refusal.delta
response.output_item.added      output_index=0 item=message   <-- 2nd, same item
response.content_part.added     content_index=1 part=output_text
response.output_text.delta
...
response.output_item.done       output_index=0 item=message   <-- only one done

The same duplication happens in the mirror order (content chunk, then refusal chunk).

After: exactly one response.output_item.added, still emitted before the first response.content_part.added; both content parts are still announced with their existing indexes.

Root cause

Each opener owns "announce the message" as well as "announce my content part". The message is only ever opened by whichever content part comes first, so the second opener must announce only its content part.

Fix and why it is minimal

Guard the ResponseOutputItemAddedEvent in both branches on the content index that was just computed two lines above: index 0 means no sibling part has opened this message yet, so this branch is the one that opens it. No new state field, no new helper, no change to content_index assignment, to the content_part.added events, to the delta events, or to the completed response. The diff is one if in each branch plus the re-indentation of the block it now wraps.

Deliberately unchanged:

  • The synthesized content-filter refusal near the end of handle_stream already runs only when neither part opened, so it keeps its single announcement.
  • Text-only and refusal-only streams (the overwhelmingly common shapes) emit a byte-identical event sequence.
  • Multi-part content_part.done emission order. In the refusal-then-text case those two events are still emitted text-first (index 1 before index 0). That is a separate ordering question from the duplicate-announcement defect fixed here, and I left it out to keep this change to one behavior. Happy to send it separately if you'd like it.

Test plan

New regression test test_stream_handler_announces_assistant_message_once_for_text_and_refusal, parametrized over both orders (refusal_then_text, text_then_refusal). It asserts one message added, one message done, matching output_index, that the single added still precedes the first content_part.added, and that both content parts are still announced at indexes 0 and 1.

  • Fails on upstream/main for the right reason, both parameters: AssertionError: assert 2 == 1 on the count of message added events.
  • Passes with this change.

Commands run from the repository root at 8b810bc4:

  • uv run pytest tests/models/test_openai_chatcompletions_stream.py -q — 89 passed (87 pre-existing + the 2 new parameters), no existing assertion changed.
  • uv run pytest -n auto --maxprocesses=9 --dist worksteal -m "not serial" -q — 6819 passed, 29 skipped.
  • make tests-serial — passed.
  • make lint (uv run ruff check) — All checks passed!.
  • make format — no reformatting needed for the changed files.
  • make typecheck (mypy + pyright) — passed.
  • git diff --check — clean.

Verification note: an earlier run of .agents/skills/code-change-verification/scripts/run.sh hit one failure in tests/extensions/memory/test_advanced_sqlite_session.py::test_branch_allocation_is_serialized_across_processes[1-shared_branch], a multiprocessing Event.wait(timeout=30) in a test unrelated to this change. It passes in isolation (4 passed) and in the full parallel suite above; it timed out only while make lint and make typecheck were saturating the machine in parallel. No code in this PR is reachable from that test.

Issue number

None — reporting and fixing directly, per the repository's guidance for small fixes. Follows up on the same both-parts scenario addressed by #4236.

Checks

  • I've added new tests, if relevant
  • I've run .agents/skills/code-change-verification/scripts/run.sh
  • I've confirmed all verification steps pass
  • If using Codex, I've run /review before submitting this PR

A Chat Completions stream whose assistant message carries both a text
part and a refusal part emitted two response.output_item.added events
for that single message, and only one response.output_item.done. Both
content-part openers unconditionally announced a fresh in-progress
assistant message, even when the sibling part had already opened the
same item at the same output_index.

Announce the message only from the opener that holds content index 0,
which is the part that actually opens it. Text-only and refusal-only
streams are unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants