`streaming=False` still emits `assistant.message_delta` through CLI 1.0.82-0

[BrandonHaynes]

Sessions created with both streaming=False and include_sub_agent_streaming_events=False still emit a root-agent assistant.message_delta event.

This contradicts the documented streaming contract and the SDK's cross-language streaming-fidelity tests, which expect assistant.message_delta to be absent when streaming is disabled. An assistant.streaming_delta event is also observed; it is included below as a related observation because the public contract is less explicit about whether this low-level network-progress event should be suppressed.

The behavior reproduces with the current stable Python SDK and both available release candidates, using each package's pinned Copilot CLI runtime:

Python SDK	Pinned CLI	Result
1.0.11	1.0.79	Emits assistant.message_delta and assistant.streaming_delta
1.0.12rc0	1.0.81-5	Emits assistant.message_delta and assistant.streaming_delta
1.0.13rc2	1.0.82-0	Emits assistant.message_delta and assistant.streaming_delta

Each turn also delivers the expected final assistant.message; the unexpected deltas are additional notifications.

Reproduction

import asyncio
import collections
import importlib.metadata
import os
import subprocess
import tempfile

import copilot
from copilot import CopilotClient, RuntimeConnection
from copilot.session import PermissionHandler


async def main() -> None:
    workdir = tempfile.mkdtemp()
    counts: collections.Counter[str] = collections.Counter()
    cli_path = os.environ["COPILOT_CLI_PATH"]

    print("sdk_version:", importlib.metadata.version("github-copilot-sdk"))
    print("sdk_module:", copilot.__file__)
    print("cli_path:", cli_path)
    print(
        "cli_version:",
        subprocess.check_output(
            [cli_path, "--no-auto-update", "--version"], text=True
        ).strip(),
    )

    client = CopilotClient(
        connection=RuntimeConnection.for_stdio(path=cli_path),
        working_directory=workdir,
        github_token=os.environ["GITHUB_TOKEN"],
        log_level="warning",
    )

    try:
        session = await client.create_session(
            model="claude-sonnet-4.6",
            streaming=False,
            include_sub_agent_streaming_events=False,
            working_directory=workdir,
            on_permission_request=PermissionHandler.approve_all,
        )
        session.on(lambda event: counts.update([event.type.value]))

        answer = await session.send_and_wait("Say the single word: ping", timeout=180)
        print("answer_type:", answer.type.value)
        print(
            "deltas:",
            {name: count for name, count in counts.items() if name.endswith("_delta")},
        )
        print("final_messages:", counts["assistant.message"])
    finally:
        await client.force_stop()


asyncio.run(main())

For each SDK version, download and select its pinned runtime before running. The runtime version was verified with copilot --no-auto-update --version before every run.

Actual behavior

All three tested combinations produced:

sdk_version: 1.0.13rc2
sdk_module: <isolated-venv>/lib/python3.12/site-packages/copilot/__init__.py
cli_path: <sdk-cache>/cli/1.0.82-0/copilot
cli_version: GitHub Copilot CLI 1.0.82-0. Run 'copilot update' to check for updates.
answer_type: assistant.message
deltas: {'assistant.message_delta': 1, 'assistant.streaming_delta': 1}
final_messages: 1

The paths and version values above are from the 1.0.13rc2 run; the other two
runs reported the corresponding SDK and pinned CLI versions shown in the table.

The observed delta events did not have a sub-agent agentId, so include_sub_agent_streaming_events=False is not expected to control them.

Expected behavior

With streaming=False, no root-agent assistant.message_delta events should be delivered to the SDK subscriber. The completed assistant.message should still be delivered.

Please also clarify whether assistant.streaming_delta is expected when streaming=False. It is documented as an ephemeral, low-level network-progress event, but the streaming-fidelity tests currently do not cover it.

Evidence for the expected contract

The SDK repository contains equivalent cross-language tests named along the lines of “should not produce deltas when streaming is disabled.” The Python test creates a session with streaming=False, asserts that no assistant.message_delta events are received, and still expects a final assistant.message.

The SDK documentation also describes streaming=True as enabling assistant.message_delta and assistant.reasoning_delta. The Rust README states explicitly that when streaming is off, only final assistant.message and assistant.reasoning events fire.

Additional context

The Python SDK appears to serialize streaming=False correctly in the session.create payload:

if streaming is not None:
    payload["streaming"] = streaming

It also serializes the independent sub-agent control correctly:

payload["includeSubAgentStreamingEvents"] = (
    include_sub_agent_streaming_events
    if include_sub_agent_streaming_events is not None
    else True
)

The SDK then forwards the runtime's session notifications. This suggests the assistant.message_delta behavior originates in the runtime, though a protocol trace would be useful to confirm it.

The current streaming-fidelity tests only reject assistant.message_delta; they do not check assistant.reasoning_delta, assistant.streaming_delta, or other streaming-only events. Expanding the tests to cover the intended complete contract could prevent similar regressions from being missed.

This report establishes the behavior for the configuration above. It does not establish that the selected model is the cause.

[github-actions]

Investigation Summary

Verdict: Confirmed bug — the bug label already applied is correct.

What I investigated

• python/copilot/client.py (create_session / resume_session methods, lines ~2617 and ~3342): The SDK correctly serializes streaming=False into the session.create JSON-RPC payload. The include_sub_agent_streaming_events flag is also serialized correctly.
• SDK event forwarding: The Python SDK does not filter events client-side — it forwards all notifications received from the CLI runtime directly to subscribers. There is no SDK-level suppression of assistant.message_delta events.
• python/e2e/test_streaming_fidelity_e2e.py: The existing test_should_not_produce_deltas_when_streaming_is_disabled test (line 48) explicitly asserts len(delta_events) == 0 when streaming=False. This test should be catching the regression if run against the live CLI.
• test/snapshots/streaming_fidelity/should_not_produce_deltas_when_streaming_is_disabled.yaml: The snapshot only records the conversation exchange; it doesn't capture runtime event emission, so the replay harness cannot detect this regression.
• nodejs/src/types.ts and dotnet/src/Types.cs: Documentation for streaming consistently states that assistant.message_delta and assistant.reasoning_delta are only emitted when streaming=true.

Root cause

The SDK is working as designed — it correctly forwards streaming=False to the CLI runtime. The problem originates in the CLI runtime itself (confirmed across versions 1.0.79, 1.0.81-5, and 1.0.82-0): when streaming=false is set in the session.create request, the runtime still emits root-agent assistant.message_delta (and assistant.streaming_delta) notifications over the JSON-RPC event stream, violating the documented contract.

Additional notes

• The streaming-fidelity E2E tests cover assistant.message_delta but currently do not assert the absence of assistant.reasoning_delta or assistant.streaming_delta when streaming=False — expanding those assertions would give broader regression coverage.
• Since the snapshot-based replay harness cannot reproduce live runtime event emission, this regression went undetected in CI. A fix in the CLI runtime is needed to suppress these delta events when streaming=false.

Generated by Bug Handler for #2450 · sonnet46 40.6 AIC · ⌖ 5.2 AIC · ⊞ 5.9K · ◷

[BrandonHaynes]

Likely a CLI issue but it's possible you want to filter on the SDK side. I'll let you folks decide.