Skip to content

perf(chat): avoid rescanning fragmented SSE carry - #7497

Closed
Platinum-Scripts wants to merge 2 commits into
lidge-jun:devfrom
Platinum-Scripts:perf/sse-fragment-carry
Closed

Platinum-Scripts wants to merge 2 commits into
lidge-jun:devfrom
Platinum-Scripts:perf/sse-fragment-carry

Conversation

@Platinum-Scripts

@Platinum-Scripts Platinum-Scripts commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Avoid splitting and re-encoding the entire unfinished Chat SSE line when the new decoded fragment contains no newline. The carry has no newline after the prior drain, so its existing UTF-8 byte count remains valid.
  • Preserve append/residual overlap reservations, cap failures, parser behavior, terminal repairs and tool-call ordering. Add deterministic work-bound and Unicode/CRLF/EOF regressions and update the owning structure docs.
  • Local actual Codex app-server integration measurements with a simulated provider found tool availability about 11.6 ms sooner for a 256 KiB frame and 115.4 ms sooner for a 1 MiB stress frame, both fragmented into 512-byte chunks (20 balanced alternating pairs per cell). Small-frame client results were inconclusive; the final small-delta adapter control was also inconclusive. These are local parser/client measurements, not vendor inference or GUI paint claims.

Verification

  • Bun 1.4.3: focused adapter regression suite passed (39 tests), including a work-bound regression that failed on the original implementation. Prior-base import-connected graph passed: 21,500 tests, 34 skipped, zero failures.
  • Current upstream dev base: 79fa3506628da65bcd599a2890f13d1cbb4a2f20. Baseline and patched adapter files each match the measured versions byte for byte.
  • Current-base bun run typecheck, bun run structure:check, bun run privacy:scan, and git diff --check passed.
  • Current-base full bun scripts/test.ts finished with exit 1: parallel lane 50,612 pass / 129 skip / 2 fail; serial lanes completed. One failure was the adapter file-size cap, corrected without raising the cap. The other, connected Desktop uses hub IDs across a cold restart (stored profile=false), reproduces in isolation on untouched upstream dev at the same base. Final focused run: 49 pass / 1 baseline failure across adapter, ratchet, and Desktop files; typecheck passed. Final import-connected graph passed: bun scripts/test.ts --changed=79fa3506628da65bcd599a2890f13d1cbb4a2f20, 21,514 pass / 34 skip / zero failures across 914 files. This draft does not attest to a passing full suite.
  • Local Bun 1.4.2 benchmarks: Final cap-safe implementation rerun: 420 adapter comparisons plus 120 actual Codex text turns and 120 actual dynamic-tool turns. Tool turns verified exact argument hashes and real tool-result replay with the original call ID. Normal default tools/cache-only search retained. No external inference. All 27 adapter cap scenarios had identical complete event sequences and budget snapshots.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. No auth, credential, dependency, or release workflow changes.

Review readiness

  • Required local validation passed; commands, results, and any full-suite exception are documented.
  • I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
  • I resolved all correct Codex and CodeRabbit findings.
  • My PR is ready for review.

Review readiness checklist

This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:

  • Required local validation passed; commands, results, and any full-suite exception are documented.

  • I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).

  • I resolved all correct Codex and CodeRabbit findings.

  • My PR is ready for review.

@coderabbitai

coderabbitai Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@github-actions github-actions Bot added the enhancement New feature or request label Oct 21, 2026
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

⏳ DRAFT

  • review readiness checklist open (0/4 boxes ticked).

What to do

  • Tick all four boxes in the PR description once you're done (currently 0/4).

Review readiness checklist

  • ⬜ Required local validation passed; commands, results, and any full-suite exception are documented.
  • ⬜ I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
  • ⬜ I resolved all correct Codex and CodeRabbit findings.
  • ⬜ My PR is ready for review.

0/4 boxes ticked.

This PR stays in draft until every box above is ticked.

@lidge-jun

Copy link
Copy Markdown
Owner

Carried together with #7497/#7498 into #7541 on current dev, with authorship and a Co-authored-by trailer kept on every commit. The carry adds a small follow-up in createSseBlockBuffer.append() that updates the cached boundary state before the budget commit. The maintainer will close this PR.

lidge-jun added a commit that referenced this pull request Oct 23, 2026
…#7497, #7498) (#7541)

* perf(chat): avoid rescanning fragmented SSE carry

Co-authored-by: Platinum-Scripts <133908028+Platinum-Scripts@users.noreply.github.com>

* perf(chat): retain carry fast path within file-size cap

Co-authored-by: Platinum-Scripts <133908028+Platinum-Scripts@users.noreply.github.com>

* perf(sse): avoid flattening fragmented block carry

Co-authored-by: Platinum-Scripts <133908028+Platinum-Scripts@users.noreply.github.com>

* fix(sse): keep block-buffer boundary state in step with the carry

Record the cached final code unit and scan eligibility as soon as the
fragment joins the buffer, before the budget commit, so a throwing commit
cannot leave them describing an older carry.

Co-authored-by: Platinum-Scripts <133908028+Platinum-Scripts@users.noreply.github.com>

---------

Co-authored-by: StreamClash Dev <dev@streamclash.local>
Co-authored-by: Platinum-Scripts <133908028+Platinum-Scripts@users.noreply.github.com>
@lidge-jun

Copy link
Copy Markdown
Owner

Triage note (2026-10-24, against dev at 2494848).

Superseded: already integrated through #7541 / c0d80ef869; do not resolve its conflicts to reland it.

Still needed on dev: no. Carried by #7541 / c0d80ef869; the fast path is present at src/adapters/openai-chat.ts:617.

Exact-head CI at triage time: 87bc972 — 28 success, 9 skipped, one cancelled historical enforce-target alongside successful replacements. Runtime ci passes in run 39674515322. lane=all on exact head: n.a.

A maintainer will decide on closing this; no action needed from you.

@lidge-jun

Copy link
Copy Markdown
Owner

Closing as superseded: this change landed on dev through the combined carry #7541 (c0d80ef869); the fast path is at src/adapters/openai-chat.ts:617. Thank you, @Platinum-Scripts — the carry keeps your authorship credit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants