fix(responses): normalize hosted search history for Codex-native continuations

[yuanyuanlove]

Summary

• Narrow ordinary canonical ChatGPT Codex /responses hosted-search projection to explicitly selected Responses Lite requests, independently of model names. Non-Lite and unknown requests retain structural hosted history. Native /responses/compact, public OpenAI API requests and other destination URLs keep their existing behavior.
• Read the final outgoing protocol intent after destination selection and existing repairs: literal true/false in x-openai-internal-codex-responses-lite wins; otherwise exact true in valid string-map client_metadata.ws_request_header_x_openai_internal_codex_responses_lite is the fallback, matching existing WS preparation. No header, metadata, model, authentication or transport behavior is changed to select Lite. The same final body feeds HTTP/SSE and WS preparation, including existing HTTP fallback.
• Within that Lite boundary, scoped bridge memo hits restore the original function call/result first; only unreferenced remaining hosted cells become labeled untrusted metadata notes. Same-input item_reference.id targets remain structural; existing store:false normalization removes references earlier. A cache miss, ID prefix, model switch, fork or restart does not establish provenance.
• Preserve client-owned raw history, non-hosted order, declarations/Lite catalogs, tool choices, schemas, call/result pairs and opaque reasoning under their existing policies. Keep the earlier bounded renderer enhancement for allowlisted top-level web_search_call.sources, including portable/routed summaries; their existing triggers and default projection are unchanged by this follow-up.
• Draft / policy review: Lite still defaults to a lossy compatibility policy, with 2,048 UTF-16 units per string, 20 entries per list and 65,536 serialized bytes of projected notes including the omission note. Converted cells lose item identity and unknown fields; notes are not fetched page content or reconstructed tool results. The bound is not a limit on original history or deliberately retained referenced cells. Explicit project-owner acceptance remains outstanding.
• Record the contributor's rationale, alternatives and evidence limits in ADR-6930, owned by the existing hosted-search continuation section. Official Codex 0.161.0 omits hosted model-tool declarations for Lite and still accepts hosted history items in its serializer. openai/codex#51962 reports restored hosted-history compaction failure and successful controls; it does not prove all ordinary Lite requests fail or hosted schema is invalid. The narrower protocol boundary preserves non-Lite/unknown continuations rather than applying universal canonical projection.
• The portable/routed summarizer renderer already present in this PR's upstream dev base originated in fix(compaction): preserve hosted search metadata as text #6746 by @yuanyuanlove and was integrated through fix(compaction): keep hosted search history as bounded reference notes in portable summaries (carry #6746) #6799 by @lidge-jun, whose amendments made the notes assistant reference text and bounded their fields, lists, and total size. Those commits are already ancestors of this PR's base; this PR extends the existing renderer for ordinary Codex continuations and preserves those bounds. It does not claim that the working native compact endpoint was missing a required repair.
• No Kimi empty-assistant fix (fix(kimi): omit explicit empty assistant history on native Responses #6931), native reasoning-retention patch, local runtime diagnostics, request captures, deployment scripts, credential or proxy/security change is included.

Verification for the upstream conflict repair (8c6a2f620dc829a812fb10aee284d72224c0c5d3)

• Merged upstream dev at dc45bb2ca92e9a4f841a73d70177bfec04722d47 into original head 231b848bbf448a04dae747108768f28d1a5bee9b without rewriting branch history. The only textual merge conflict was the initial entries in tests/fixtures/test-layout-expected.json; the resolution retains this PR's two hosted-search tests and upstream's four service-health-sentinel tests. The explicit/golden maps remain identical at 2,224 entries, and all 2,244 tracked test files resolve without loss or misplaced files.
• The merged structure/providers-and-adapters.md reached 602 lines against its existing 600-line budget. Reflowed the existing hosted-search bridge paragraph from three physical lines to one, preserving every word and the existing budget. Independent read-only review confirms both sides of all seven overlapping files, the upstream empty-assistant implementation/order, and the Lite-only policy and documentation are preserved. The remaining owner-acceptance decision is unchanged.
• Final-tree focused validation with pinned Bun 1.4.0: 144 pass / 0 fail, 1,373 assertions:
  • bun test --isolate tests/adapters/openai/openai-native-hosted-search-history.test.ts tests/adapters/openai/openai-empty-assistant-history.test.ts tests/test-layout.test.ts tests/test-layout-tooling.test.ts: 80 pass, 1,030 assertions.
  • bun test --isolate tests/responses/openai-responses-passthrough.test.ts -t 'strips invalid type-specific ids from serialized input items': 1 pass, 16 assertions; 184 filtered out.
  • bun test --isolate tests/codex-integration/codex-metadata-integrity.test.ts -t 'Codex request transport metadata|Lite follows the serialized wire model': 12 pass, 224 assertions; 12 filtered out, including the separate network-listening cases.
  • bun test --isolate tests/ci-workflows/structure-ssot.test.ts: 51 pass, 103 assertions.
• bun run typecheck, bun run privacy:scan, final bun run structure:check, and bun scripts/file-size-ratchet.ts passed. Working-tree diff checks and the final PR delta against merged dev pass. The merge-relative delta from the old PR head includes nine existing upstream blank-at-EOF warnings; those unrelated upstream files remain byte-identical and were not cleaned up.
• In docs-site/, bun install --frozen-lockfile and bun run build passed with a workspace-local XDG config directory and Astro telemetry disabled: 569 pages / 79,359 internal links. Existing large-chunk, missing i18n/404-entry and sshconfig syntax-highlighting warnings remain. No tracked lockfile, dependency, generated-output or configuration change was introduced by validation.
• Published through Git data with the original-head lease and force:false. Refetched the fork branch and verified the complete tree f83df41ee8ab0850c81c975101c528d8d17b5a23, ordered parents 231b848bbf448a04dae747108768f28d1a5bee9b and dc45bb2ca92e9a4f841a73d70177bfec04722d47, exact equality with the tested staged tree, and a clean published worktree. GitHub now reports mergeable:true; the PR remains Draft and blocked on readiness.
• Exact-head Cross-platform CI 38253795370, React Doctor 38253795436 and Catalog async contracts 38253795442 are action_required. Metadata checks at this update: hygiene: queued; label: completed / success; resolve-pr: queued. Queued metadata and older-head receipts are not current runtime-CI passes; no workflow approval was granted.
• The full suite and mock-network server/native-compact suites were not rerun in this restricted cloud environment. Previously denied opaque-blob/compaction-override/raw-proxy and sensitive WS paths were not retried or bypassed; no proxy, security, permission or credential settings were changed. Ordinary live HTTP/WS, backend acceptance and sustained original-conversation recovery remain unverified. Explicit project-owner acceptance of the remaining default Lite loss policy remains outstanding; the follow-up source review is not approval.

Prior Lite-only follow-up verification (231b848bbf448a04dae747108768f28d1a5bee9b)

• Baseline: f9bcaefcaf6214567e54a73dae101a2c51ce16f4, a one-line docs correction over the previously reviewed f093002f5caff80203e83f44250c621891101cab. Upstream base remains 9b6f4b2632492b3df637dccc009d6a01ed47c85d; no merge/rebase was needed. This follow-up changes one production file, two test files and related documentation; the shared renderer, native compact implementation and transport marker handling have no diff from that baseline.
• Published as an ordinary update with the original-head lease and no force. Refetched the fork branch and verified the complete tree 32b51b94c0aa340b294fc2f14eaa63c5f9189ff0, sole parent f9bcaefcaf6214567e54a73dae101a2c51ce16f4, and exact equality with the staged/working files.
• Test-first: the universal baseline failed nine new non-Lite/unknown boundary cases before the Lite gate; final focused tests pass with pinned Bun 1.4.0:
  • bun test --isolate tests/adapters/openai/openai-native-hosted-search-history.test.ts: 41 pass / 0 fail, 392 assertions.
  • bun test --isolate tests/responses/openai-responses-passthrough.test.ts -t 'strips invalid type-specific ids from serialized input items': 1 pass / 0 fail, 16 assertions; restores the non-Lite structural hosted-ID control.
  • bun test --isolate tests/codex-integration/codex-metadata-integrity.test.ts -t 'Codex request transport metadata|Lite follows the serialized wire model': 12 pass / 0 fail, 224 assertions. Filters exclude the separate listening/network test.
• These 54 tests / 632 assertions cover HTTP true/false/absent and mixed-case names, exact flag values, valid/invalid/absent WS metadata, conflicts and configured-header precedence, malformed metadata with explicit HTTP true, other destinations/public API, native-compaction flags, existing portable builder behavior, bridge restoration/misses, same-input reference exemptions, raw/cipher preservation, bounded sources and 190 client call/result pairs. They are adapter/WS-preparation fixtures, not live HTTP/WS E2E or backend acceptance evidence.
• bun run typecheck, bun run privacy:scan, bun run structure:check, bun scripts/file-size-ratchet.ts, and working/staged git diff --check passed.
• In docs-site/, bun install --frozen-lockfile and bun run build passed using a workspace-local XDG config directory with Astro telemetry disabled: 569 pages / 79,294 internal links. Existing large-chunk and missing i18n/404-entry warnings remain; there is no lockfile/dependency change.
• Independent read-only source review checked the gate against existing HTTP/WS precedence, endpoint/compaction exclusions, restoration order, references, tests and documentation. It provides no project-owner approval, CI authorization or live acceptance evidence.
• Exact-head GitHub status: Cross-platform CI 38123897992, React Doctor 38123897991, and Catalog async contracts 38123898071 are action_required. Metadata checks: enforce-target: success; hygiene: success; resolve-pr: success; label: success. These are not runtime CI passes; no workflow authorization was granted.
• Full-suite and mock-network server suites were not rerun in this restricted cloud environment. The previously denied fixture/raw-proxy path was not retried or bypassed; no proxy/security settings were changed. Sustained recovery of the original conversation, live ordinary HTTP/WS, public API and cross-platform acceptance remain outstanding.
• Official OpenAI documentation treats hosted search as valid output and describes manually replaying response output: web search, conversation state. These public API contracts do not independently establish private Codex backend behavior. The PR remains Draft.

Prior revision receipts

The earlier author-reported broad import-connected run (3,939 pass / 1 skip; 157 files), focused 313-test and 224-test runs, and mocked native-compact send guards describe the superseded universal-gate revision f093002f5caff80203e83f44250c621891101cab. They were not rerun for this Lite-only follow-up and are not presented as current-head full-suite or live acceptance results.

Checklist

• Scope stays focused and avoids unrelated cleanup.
• Docs or release notes were updated when needed.
• Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. (No credential/auth/workflow/dependency policy change; untrusted metadata only, synthetic fixtures, no invented replay scope.)

Co-authored-by: JUN jun@lidgeai.com

Review readiness checklist

This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:

• Required local validation passed; commands, results, and any full-suite exception are documented.

• I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).

• I resolved all correct Codex and CodeRabbit findings.

• My PR is ready for review.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

✅ Deterministic PR hygiene checks passed.

[github-actions]

⏳ DRAFT

• review readiness checklist open (0/4 boxes ticked).

What to do

• Tick all four boxes in the PR description once you're done (currently 0/4).

Review readiness checklist

• ⬜ Required local validation passed; commands, results, and any full-suite exception are documented.
• ⬜ I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
• ⬜ I resolved all correct Codex and CodeRabbit findings.
• ⬜ My PR is ready for review.

0/4 boxes ticked.

This PR stays in draft until every box above is ticked.

[Ingwannu]

Source review of exact head f093002f5caff80203e83f44250c621891101cab against 9b6f4b2632492b3df637dccc009d6a01ed47c85d (16 files). This is not approval or CI authorization.

The ordinary projection is selected by the exact canonical Codex /responses URL, after bridge restoration and existing repair, and excludes native/portable compaction. Same-input referenced hosted cells stay structural; the shared note renderer bounds only converted notes, not all request input. Top-level sources gain the same allowlist/bounds as action sources. The new fixtures explicitly exercise public-API exclusion, reference handling, memo hits/misses, raw-input immutability, ordering, Lite declarations, repeated turns, and native compact preservation. I also checked the changed existing expectations rather than treating them as unchanged regression evidence.

The important acceptance decision is behavioral: every ordinary canonical Codex request opts into lossy conversion, including valid native hosted history, not just a diagnosed invalid cell or an operator-enabled workaround. OpenAI's web-search guide documents structural web_search_call output, and its conversation-state guide describes replaying complete output. Those public API contracts do not independently prove what the private Codex backend accepts. The PR correctly avoids calling this schema rejection, but fixture success cannot establish the universal live compatibility/quality outcome. The project owner needs to explicitly accept this default loss policy before readiness; do not infer source provenance from a cache miss or ID prefix.

One factual docs correction is needed: docs-site/src/content/docs/zh-cn/reference/proxy-formats.md calls this a policy explicitly chosen by the user (用户明确选择). There is no user opt-in in this patch. Please describe it as the implementation's explicit/default lossy policy, or actually add an owner-approved opt-in design; do not imply a control that does not exist. Keep the rationale for universal projection versus an opt-in or narrower repair recorded in the appropriate existing decision documentation.

I have not run these new tests or live Codex conversations; the large test/typecheck/docs receipts in the description remain author evidence. Current-head Cross-platform 38115457766, Catalog 38115457751 and React Doctor 38115457777 are action_required. Latest target/hygiene checks pass, superseding earlier metadata failures, but they are not runtime CI. Please retain Draft and the stated coverage limits. No scan, workflow authorization, sponsorship, native/provider call or contributor-branch change.

[yuanyuanlove]

Follow-up to this review: corrected the Chinese reference wording in f9bcaef to “当前实现默认采用的有损历史策略”. It now describes the implementation default without implying an end-user opt-in. The commit changes only that sentence; runtime behavior and defaults are unchanged.

For this docs-only follow-up, Bun 1.4.0 bun install --frozen-lockfile passed, and bun run build passed with a workspace-local XDG config directory and telemetry disabled: 569 pages and 79,294 internal links checked. The first build attempt stopped at an unavailable global telemetry config directory. Working/staged git diff --check passed. After publication, I fetched the branch and verified the full tree, the single original-head parent, and the exact one-line delta. No runtime or live conversation tests were rerun for this wording-only change.

Explicit project-owner acceptance of the default lossy policy and an owner-confirmed decision record explaining universal projection versus an opt-in or narrower repair remain outstanding. The PR remains Draft, with the existing coverage limits. Exact-head target/hygiene checks pass, while Cross-platform CI (38120058888), Catalog async contracts (38120058867), and React Doctor (38120058819) are action_required. This correction provides no new evidence of live Codex acceptance, sustained recovery of the original conversation, or resolution of all 502s.

[yuanyuanlove]

Follow-up to this review: narrowed the ordinary history policy in 231b848bbf448a04dae747108768f28d1a5bee9b. Only exact canonical Codex /responses requests with explicit Responses Lite intent project remaining unreferenced hosted-search cells. Non-Lite and unknown requests retain structural hosted history.

The final HTTP Lite header's literal true/false is authoritative; otherwise exact true in valid string-map frame metadata is the fallback, matching existing WS preparation. This reads current protocol intent without changing headers, metadata or transport behavior. It also preserves the existing HTTP fallback semantics. Models, subscription authentication, missing hosted declarations and additional_tools alone do not select the policy. Bridge restoration still runs first, same-input references retain their exemption, and native compact, public API and portable/routed summary triggers are unchanged.

The rationale and alternatives are now recorded in ADR-6930, linked from the existing contract owner. Official Codex 0.161.0 omits hosted model-tool declarations for Lite but still replays hosted history. openai/codex#51962 supplies restored-history compaction and control observations, not proof that every ordinary Lite request fails. The protocol-specific narrowing therefore avoids the previous universal canonical policy without pretending to classify historical issuers. The record states the contributor's implementation rationale; explicit project-owner acceptance of the remaining default Lite loss policy is still outstanding.

Current follow-up validation: 54 passing fixture tests, 632 assertions, covering protocol precedence/conflicts, missing/malformed markers, mixed-case headers, configured headers, WS preparation and preserved boundaries. TypeScript, privacy, structure, file-size and diff checks passed; English/Chinese docs build passed with 569 pages and 79,294 internal links. Independent read-only source review found no blocking code inconsistency. I refetched the published branch and verified the full tree and sole original-head parent.

The updated description separates these results from the earlier superseded revision's broad test receipts. No full-suite, denied mock-network path or live HTTP/WS/backend acceptance test was rerun; no proxy/security settings were changed. All four exact-head metadata checks pass, while Cross-platform CI (38123897992), React Doctor (38123897991) and Catalog async contracts (38123898071) remain action_required. The PR remains Draft; this is not readiness, owner acceptance or evidence that all 502s are resolved.

[Ingwannu]

Follow-up on my earlier review, current head 231b848bbf448a04dae747108768f28d1a5bee9b.

The new final-request Lite predicate addresses the earlier universal-canonical scope objection: literal HTTP true/false wins, valid string-map metadata supplies the fallback, and unknown/non-Lite requests retain structural hosted history. The added precedence/conflict and absent/malformed marker fixtures exercise that restriction. The Chinese wording no longer describes a nonexistent user opt-in, and ADR-6930 now explicitly records the contributor's rationale and alternatives. Those specific requested changes are addressed by source/document inspection.

The remaining choice is still substantive: ordinary Lite requests default to a lossy history projection, not merely a repair proven necessary for every request. The ADR correctly does not claim project-owner acceptance. Keep Draft pending that decision and current required reviews/CI. Author fixture/build receipts and replay/control observations are not independently executed live backend or sustained-conversation acceptance; I did not rerun the fixtures, call a provider or run a scan in this pass. This follow-up is not approval of the remaining policy or a claim that #6740/#6969 are fixed.

[yuanyuanlove]

Follow-up to the latest source review: 8c6a2f620dc829a812fb10aee284d72224c0c5d3 now incorporates upstream dev dc45bb2ca92e9a4f841a73d70177bfec04722d47. The sole textual conflict was the test-layout golden roster; both hosted-search tests and all four upstream service-health-sentinel entries are retained. The combined provider/adapter document also exceeded its existing 600-line budget by two, resolved by reflowing an existing bridge paragraph without changing its text or the budget.

Independent read-only review confirmed preservation of the Lite predicate, Chinese wording and ADR addressed in your follow-up, plus bridge-restoration order, item-reference exemptions, native/portable boundaries and upstream empty-assistant behavior. The PR's policy has no new changes or opt-in control.

Validation on the final merged tree: 144 passing focused tests / 1,373 assertions, comprising hosted-search/empty-assistant adapters and layout guards (80), the structural hosted-ID control (1), filtered transport-marker/WS-preparation fixtures (12), and structure SSOT tests (51). TypeScript, privacy, structure and file-size gates pass. The docs build passes with 569 pages and 79,359 internal links. The PR delta against merged dev is whitespace-clean; nine blank-at-EOF warnings belong to untouched upstream files in the merge-relative delta. Full-suite, denied mock-network/raw-proxy paths, live HTTP/WS/backend acceptance and sustained conversation recovery were not rerun or claimed.

The ordinary fork update used the original-head lease with no force. I refetched it and verified complete tree f83df41ee8ab0850c81c975101c528d8d17b5a23, ordered parents 231b848bbf448a04dae747108768f28d1a5bee9b / dc45bb2ca92e9a4f841a73d70177bfec04722d47, exact agreement with the tested tree, and a clean worktree. GitHub now reports the conflict cleared (mergeable:true). Exact-head Cross-platform CI, React Doctor and Catalog async contracts remain action_required; metadata jobs are queued at this observation. No workflow approval was granted.

The PR remains Draft. Explicit project-owner acceptance of the remaining default Lite loss policy, current required CI and live acceptance coverage remain outstanding. This maintenance update adds no approval or claim that #6740/#6969 are fixed.

[yuanyuanlove]

Upstream conflict-maintenance follow-up to the latest source review: after the first repair, dev advanced to 3e58c2ce3ee5433901ed467fe87c32fb4b5c5e77 with #6962 and the PR became conflicted again. af3419d88bb39cbce7141e4970628e4a8cd72572 incorporates that commit without rewriting branch history.

The sole new conflict was scripts/test-layout/layout.json. Its resolution keeps this PR's two hosted-search entries and all six new upstream billing entries. The golden roster merged automatically; both duplicate-free maps now contain the full 2,230-entry union. Independent read-only review confirmed exact parent-blob/mode preservation for all 13 PR-only and 26 upstream-only paths, exact disjoint-hunk unions in the shared documents, and unchanged Lite-only predicate, restoration order, same-input reference exemption, and native/portable boundaries. The provider/adapter document remains within its existing 600-line budget. There is no new policy, opt-in, credential, permission or workflow change beyond the preserved upstream commit.

Validation on the final merged tree: 144 focused tests pass / 0 fail, 1,373 assertions. Hosted-search/empty-assistant adapters, layout and structure tests account for 131; the filtered structural hosted-ID control accounts for 1, and filtered transport-marker/WS-preparation fixtures for 12. bun run typecheck, bun run privacy:scan, bun run structure:check, file-size ratchet, working-tree checks and both latest-dev/previous-head delta checks pass. The docs build was rerun because the upstream providers guide changed: 569 pages and 79,362 internal links pass, with the existing warnings retained. Full-suite, denied mock-network/raw-proxy paths and live backend/conversation acceptance remain unrun.

I refetched the ordinary fork update and verified complete tree 9cc51d88480fe64391be7cf4f18ddf3a1b450b49, ordered parents 8c6a2f620dc829a812fb10aee284d72224c0c5d3 / 3e58c2ce3ee5433901ed467fe87c32fb4b5c5e77, exact equality with the tested tree, and a clean worktree. GitHub again reports mergeable:true. Exact-head Cross-platform CI, React Doctor and Catalog async contracts are action_required; metadata jobs are queued/pending at this observation. No workflow approval was granted.

The PR remains Draft. Explicit project-owner acceptance of the remaining default Lite loss policy and current required CI remain outstanding; this repair supplies no merge approval, live acceptance evidence or claim that #6740/#6969 are resolved. The description's 8c6a2f6 verification remains a dated prior-head receipt; this comment records the new repair.

[Ingwannu]

Follow-up source review pinned to the newly advanced head af3419d88bb39cbce7141e4970628e4a8cd72572, not the earlier 8c6a2f6 receipt. I fetched and verified its ordered parents (8c6a2f620d, current dev 3e58c2ce3e). The hosted-search projection implementation, passthrough and ADR-6930 are unchanged by this second merge. The first merge's passthrough addition was dev's empty-assistant normalization, not a new change to the Lite predicate.

The author's 144-test merged-tree receipts are not independent execution by me. No local rerun while foreign Cargo builds are active. Owner acceptance of the default lossy Lite policy, current required CI and live/denied-path acceptance remain outstanding. I have not authorized old fork workflows: the user's automatic privacy-CI exception applies only to newly created PRs. This stays a scoped COMMENT, not approval, sponsorship, a scan result or a claim that #6740/#6969 is fixed.