Skip to content

fix(stream): distinguish an API stream drop from a clean end / user Esc (#133) - #146

Merged
justrach merged 1 commit into
mainfrom
fix/kimi-stream-drop-133
Jul 10, 2026
Merged

justrach merged 1 commit into
mainfrom
fix/kimi-stream-drop-133

Conversation

@justrach

Copy link
Copy Markdown
Owner

Closes #133.

Problem

kimi-k2.6 (Moonshot, an OpenAI-compatible chat/completions provider) drops its SSE stream mid-generation — often mid-reasoning, before any output. The reporter saw "× interrupted (esc)" on 0.0.187, which predates the #134 fixes. Those fixes stopped an idle stall from being mislabeled as Esc, but left a residual gap for .openai-kind providers:

Once body bytes arrived (got_body), any later read close/reset was treated as a clean end (break :stream). isStreamEnd recognizes only the literal data: [DONE] sentinel for .openai, so a drop before [DONE] was silently accepted — and assembleOpenAI defaults the missing finish_reason to "stop", so a truncated answer was saved as a complete, successful turn with no signal at all (worse than the original symptom).

Fix — distinguish "confirmed complete" from "connection died"

  • Add saw_done, set only when the provider's terminal event is actually observed: [DONE]/structural terminator (via isStreamEnd), or — for .openai — a non-null finish_reason chunk (openaiComplete), recorded without breaking early so a trailing usage chunk + [DONE] still land (no cost/usage regression).
  • The three post-body read-close sites now break cleanly only if saw_done; a close with body but no terminal returns the new error.StreamDropped and prints ⚠ connection dropped — response ended early.
  • error.StreamDropped is handled like a stall (no retry — the partial already streamed): agent_request records it + telemetry; mainloop/repl_glue keep the partial with [response ended early: connection dropped] (never [response interrupted by user]) and emit the structured --json error.

Safety

Genuine Esc handling is untouched (escPressed / esc_cancel / the stall watchdog). Documented behavior change: an OpenAI-compatible gateway that ends a stream with neither [DONE] nor a finish_reason (nonstandard) is now reported as a drop instead of silently succeeding — the partial is preserved either way.

Verified

zig build · zig build test 151/151 (new isStreamEnd/openaiComplete cases lock the drop-vs-clean-end distinction, incl. the kimi mid-reasoning case and an escaped-finish_reason-in-content guard) · a live codegraff/deepseek (.openai) turn still completes cleanly with no false "connection dropped".

Note: the original "interrupted (esc)" symptom is already fixed on main by the #134 commits; this closes the remaining silent-truncation gap for Moonshot-style mid-stream drops.

🤖 Generated with Claude Code

…sc (#133)

kimi-k2.6 (Moonshot, an OpenAI-compatible chat/completions provider) drops
its SSE stream mid-generation — often mid-reasoning, before any output. The
#134 fixes already stopped an idle-stall from being mislabeled "interrupted
(esc)", but left a residual gap for `.openai`-kind providers: once body bytes
had arrived (`got_body`), ANY later read close/reset was treated as a clean
end (`break :stream`). Since `isStreamEnd` recognizes only the literal
`[DONE]` sentinel for `.openai`, a drop before `[DONE]` was silently accepted,
and `assembleOpenAI` defaults the missing `finish_reason` to "stop" — so a
truncated answer was saved as a complete, successful turn with no signal.

Distinguish "confirmed complete" from "connection died":
- Add `saw_done`, set only when the provider's terminal event is actually seen
  — `[DONE]`/structural terminator (isStreamEnd), or, for `.openai`, a non-null
  `finish_reason` chunk (openaiComplete) recorded WITHOUT breaking early so a
  trailing usage chunk + `[DONE]` still arrive.
- The three post-body read-close sites now `break` only if `saw_done`; a close
  with body-but-no-terminal returns the new `error.StreamDropped` and prints
  "⚠ connection dropped — response ended early".
- Handle `error.StreamDropped` like a stall (no retry — the partial already
  streamed): agent_request records it + telemetry; mainloop/repl_glue keep the
  partial with "[response ended early: connection dropped]" (never "[response
  interrupted by user]") and emit the structured --json error.

Real Esc handling is untouched (escPressed / esc_cancel / the stall watchdog).
Behavior change to note: an OpenAI-compatible gateway that ends a stream with
neither `[DONE]` nor a `finish_reason` (nonstandard) is now reported as a drop
instead of silently succeeding — the partial is preserved either way.

Verified: zig build; zig build test 151/151 (new isStreamEnd/openaiComplete
cases for the drop-vs-clean-end distinction); a live codegraff/deepseek
(.openai) turn still completes cleanly with no false "connection dropped".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@justrach
justrach merged commit 7ec8129 into main Jul 10, 2026
6 checks passed
@justrach
justrach deleted the fix/kimi-stream-drop-133 branch July 15, 2026 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

kimi-k2.6 stream drops mid-reasoning (3rd occurrence, 160K context, 76% window)

1 participant