Repository navigation
fix(stream): distinguish an API stream drop from a clean end / user Esc (#133) - #146
Merged
Merged
Conversation
…sc (#133) kimi-k2.6 (Moonshot, an OpenAI-compatible chat/completions provider) drops its SSE stream mid-generation — often mid-reasoning, before any output. The #134 fixes already stopped an idle-stall from being mislabeled "interrupted (esc)", but left a residual gap for `.openai`-kind providers: once body bytes had arrived (`got_body`), ANY later read close/reset was treated as a clean end (`break :stream`). Since `isStreamEnd` recognizes only the literal `[DONE]` sentinel for `.openai`, a drop before `[DONE]` was silently accepted, and `assembleOpenAI` defaults the missing `finish_reason` to "stop" — so a truncated answer was saved as a complete, successful turn with no signal. Distinguish "confirmed complete" from "connection died": - Add `saw_done`, set only when the provider's terminal event is actually seen — `[DONE]`/structural terminator (isStreamEnd), or, for `.openai`, a non-null `finish_reason` chunk (openaiComplete) recorded WITHOUT breaking early so a trailing usage chunk + `[DONE]` still arrive. - The three post-body read-close sites now `break` only if `saw_done`; a close with body-but-no-terminal returns the new `error.StreamDropped` and prints "⚠ connection dropped — response ended early". - Handle `error.StreamDropped` like a stall (no retry — the partial already streamed): agent_request records it + telemetry; mainloop/repl_glue keep the partial with "[response ended early: connection dropped]" (never "[response interrupted by user]") and emit the structured --json error. Real Esc handling is untouched (escPressed / esc_cancel / the stall watchdog). Behavior change to note: an OpenAI-compatible gateway that ends a stream with neither `[DONE]` nor a `finish_reason` (nonstandard) is now reported as a drop instead of silently succeeding — the partial is preserved either way. Verified: zig build; zig build test 151/151 (new isStreamEnd/openaiComplete cases for the drop-vs-clean-end distinction); a live codegraff/deepseek (.openai) turn still completes cleanly with no false "connection dropped". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #133.
Problem
kimi-k2.6 (Moonshot, an OpenAI-compatible chat/completions provider) drops its SSE stream mid-generation — often mid-reasoning, before any output. The reporter saw "× interrupted (esc)" on 0.0.187, which predates the #134 fixes. Those fixes stopped an idle stall from being mislabeled as Esc, but left a residual gap for
.openai-kind providers:Once body bytes arrived (
got_body), any later read close/reset was treated as a clean end (break :stream).isStreamEndrecognizes only the literaldata: [DONE]sentinel for.openai, so a drop before[DONE]was silently accepted — andassembleOpenAIdefaults the missingfinish_reasonto"stop", so a truncated answer was saved as a complete, successful turn with no signal at all (worse than the original symptom).Fix — distinguish "confirmed complete" from "connection died"
saw_done, set only when the provider's terminal event is actually observed:[DONE]/structural terminator (viaisStreamEnd), or — for.openai— a non-nullfinish_reasonchunk (openaiComplete), recorded without breaking early so a trailing usage chunk +[DONE]still land (no cost/usage regression).breakcleanly only ifsaw_done; a close with body but no terminal returns the newerror.StreamDroppedand prints⚠ connection dropped — response ended early.error.StreamDroppedis handled like a stall (no retry — the partial already streamed):agent_requestrecords it + telemetry;mainloop/repl_gluekeep the partial with[response ended early: connection dropped](never[response interrupted by user]) and emit the structured--jsonerror.Safety
Genuine Esc handling is untouched (
escPressed/esc_cancel/ the stall watchdog). Documented behavior change: an OpenAI-compatible gateway that ends a stream with neither[DONE]nor afinish_reason(nonstandard) is now reported as a drop instead of silently succeeding — the partial is preserved either way.Verified
zig build·zig build test151/151 (newisStreamEnd/openaiCompletecases lock the drop-vs-clean-end distinction, incl. the kimi mid-reasoning case and an escaped-finish_reason-in-content guard) · a livecodegraff/deepseek(.openai) turn still completes cleanly with no false "connection dropped".Note: the original "interrupted (esc)" symptom is already fixed on
mainby the #134 commits; this closes the remaining silent-truncation gap for Moonshot-style mid-stream drops.🤖 Generated with Claude Code