Skip to content

fix(exports): preserve split UTF-8 diagnostics - #1249

Merged
mldangelo-oai merged 3 commits into
openai:mainfrom
sylvesterkaczmarek:fix/preserve-helper-diagnostic-utf8
Oct 5, 2026
Merged

mldangelo-oai merged 3 commits into
openai:mainfrom
sylvesterkaczmarek:fix/preserve-helper-diagnostic-utf8

Conversation

@sylvesterkaczmarek

@sylvesterkaczmarek sylvesterkaczmarek commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Export-helper diagnostics can be corrupted when stderr splits a UTF-8 character between pipe chunks. Decode the stream continuously so returned diagnostics and helper errors preserve the original text.

Changes

  • Use stderr's built-in UTF-8 decoder before collecting the existing diagnostic tail.
  • Add two deterministic cases to the existing export tests: successful and failing helpers, with text delivered one byte at a time across two-, three-, and four-byte characters.

Testing

  • All 6 tests in artifact-export.test.ts passed.
  • Both new regression cases fail with the old decoder and pass with the fix.
  • Independent real Python subprocess probes reproduced the corruption and verified the fix under Node and Bun.
  • pnpm run types, pnpm run format, plugin assembly, and git diff --check passed.
  • Full SDK runs completed with seed 12345 (3,536 passed, 36 failed) and random seed 136913072 (3,561 passed, 11 failed), with 53 skips each. The failures came from fixture permissions and enclosing Git markers in the local test environment.
  • All six affected test files passed in a clean checkout with umask 022: 378 tests passed.
  • Required Linux, macOS, and Windows CI checks passed. Local test runs used NODE_NO_WARNINGS=1 to suppress the Node proxy warning.

Risk and rollout

Small decoding correction using the stream's existing support for split characters. No public interface changes. The regression tests use in-memory streams and restore the subprocess spy after each case.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

Signed-off-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com>
@github-actions github-actions Bot added the bug Something isn't working label Oct 5, 2026

@mldangelo-oai mldangelo-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this fix. I reproduced the UTF-8 corruption with a real helper process; using the stream decoder is the right small change.

I simplified the regression coverage into two cases in the existing export test file. They deliver mixed ASCII and multibyte text one byte at a time and check both returned diagnostics and thrown errors. Both fail with the old decoder and pass with the fix, without the subprocess handshake and polling machinery.

All six export tests, types, and formatting pass. This is worth keeping; the broader SDK checks and CI will finish before merge.

@mldangelo-oai
mldangelo-oai merged commit 1345ea3 into openai:main Oct 5, 2026
55 checks passed
@github-actions github-actions Bot mentioned this pull request Oct 5, 2026
3 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants