Python: Support prompt cache breakpoints for GPT-5.6 models in OpenAI clients#7163
Merged
eavanvalkenburg merged 7 commits intoJul 22, 2026
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds Python OpenAI client support for GPT‑5.6 prompt caching controls by (a) exposing request-level prompt_cache_options in the typed options and (b) forwarding per-part prompt_cache_breakpoint metadata from Content.additional_properties into the outgoing request payloads, while preserving existing message-shape behavior when no breakpoint is present.
Changes:
- Introduces
PromptCacheOptionsand a sharedattach_prompt_cache_breakpoint(...)helper in_shared.py. - Adds
prompt_cache_optionsto both Responses (OpenAIChatOptions) and Chat Completions (OpenAIChatCompletionOptions) typed option sets. - Updates Chat Completions content/message preparation to keep list-form text parts when a breakpoint is present, and adds unit tests covering breakpoint propagation and options passthrough.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| python/packages/openai/agent_framework_openai/_shared.py | Adds a typed PromptCacheOptions shape and a helper to copy prompt_cache_breakpoint from Content.additional_properties into outgoing parts. |
| python/packages/openai/agent_framework_openai/_chat_completion_client.py | Plumbs breakpoint copying into Chat Completions parts and adjusts string-vs-list flattening so breakpoints are not lost. |
| python/packages/openai/agent_framework_openai/_chat_client.py | Plumbs breakpoint copying into Responses API parts and adds typed support for prompt_cache_options. |
| python/packages/openai/tests/openai/test_openai_chat_completion_client.py | Adds unit tests ensuring breakpoints are preserved (list-form) and prompt_cache_options passes through. |
| python/packages/openai/tests/openai/test_openai_chat_client.py | Adds unit tests ensuring breakpoint propagation for Responses parts and prompt_cache_options passthrough. |
Contributor
Author
@microsoft-github-policy-service agree |
Contributor
Python Test Coverage Report •
Python Unit Test Overview
|
||||||||||||||||||||||||||||||||||||||||
eavanvalkenburg
left a comment
eavanvalkenburg
left a comment
Member
There was a problem hiding this comment.
would be good to have a sample for this as well
Contributor
Author
|
Added a sample: |
giles17
added
the
documentation
Mordris
force-pushed
the
feat/openai-prompt-cache-breakpoints-7157
branch
from
July 22, 2026 06:54
50f9932 to
eafcc86
Compare
Mordris
force-pushed
the
feat/openai-prompt-cache-breakpoints-7157
branch
from
July 22, 2026 07:11
eafcc86 to
33863fa
Compare
… clients Add request-level prompt_cache_options to OpenAIChatOptions and OpenAIChatCompletionOptions, and forward a per-part prompt_cache_breakpoint from Content.additional_properties onto the content blocks each API supports. Text parts that carry a breakpoint keep typed list content, since the plain-string form cannot hold one; without a breakpoint the existing string forms are unchanged.
Replace the custom PromptCacheOptions TypedDict with the openai SDK's own types for each API, which raises the openai floor to 2.45.0 where those types were introduced. Make the breakpoint helper private to the two chat clients. Add a prompt caching sample with a README entry, and unquote the helper's Content annotation so the pyupgrade hook passes.
The SDK's PromptCacheOptions types only exist in openai 2.45.0 and later, so each client falls back to a local mirror when the import fails and the dependency floor stays at 2.25.0. A TYPE_CHECKING-only import is not enough because the options classes are introspected with get_type_hints() at runtime. Verified against openai 2.25.0: the package imports, the fallback resolves, and part-level breakpoints still work; sending the option itself requires 2.45.0, which the field docstrings now note.
Assigning None instead, as suggested in review, trips pyright's reportInvalidTypeForm on the field annotation (the symbol becomes type | None after the try/except). An empty TypedDict gives the same effect for users on older openai versions: any content they put in prompt_cache_options is flagged by their type checker, since the option cannot be sent on those versions anyway, while get_type_hints() on the options classes keeps working at runtime.
…k type The empty-TypedDict fallback flagged valid `prompt_cache_options` usage under pyright on every openai version — including this PR's own `client_prompt_caching.py` sample (`poe check -S`) — because pyright resolves the try/except symbol to the fallback shape regardless of the installed openai, while mypy/ty resolve the failed import to `Any` and never warn. So a type-only "warn on old openai" signal is not achievable cleanly across type checkers. Restore the faithful fallback (mirrors the SDK's `mode`/`ttl` shape) so the option type-checks identically on every supported openai version, and add a runtime guard: setting `prompt_cache_options` on openai < 2.45 now raises a clear ChatClientInvalidRequestException instead of forwarding an unusable option to the SDK. This keeps the option non-silent for all users regardless of type checker, without forcing an openai upgrade. Adds tests covering the guard for both clients.
Mordris
force-pushed
the
feat/openai-prompt-cache-breakpoints-7157
branch
from
July 22, 2026 07:22
33863fa to
69f8e10
Compare
The system/developer branch switched to list-form content whenever prompt_cache_breakpoint was set to any non-None value, but the option is only attached when the value is a mapping. A malformed value (e.g. a string) therefore changed the message shape without adding a breakpoint. Decide the shape from the built part instead, matching the user-role path.
eavanvalkenburg
approved these changes
Jul 22, 2026
eavanvalkenburg
enabled auto-merge
July 22, 2026 15:27
TaoChenOSU
approved these changes
Jul 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation & Context
GPT-5.6 models support explicit prompt cache breakpoints, and cache writes are billed on these models, so controlling where a prefix is cached now matters for cost. Today the clients silently drop a
prompt_cache_breakpointset throughContent.additional_properties, andprompt_cache_optionsis not part of the typed chat options, so neither knob is usable from the framework.Description & Review Guide
What are the major changes?
prompt_cache_optionsfield onOpenAIChatOptionsandOpenAIChatCompletionOptions, typed with the openai SDK's ownPromptCacheOptionsparams type for each API._prepare_optionsalready forwards options generically, so no extra plumbing was needed._attach_prompt_cache_breakpointhelper (private to the two chat clients) copiesprompt_cache_breakpointfromContent.additional_propertiesonto the outgoing part, following the existingdetail/file_id/filenamepattern. It is applied to the blocks each API accepts:input_text/input_image/input_filefor the Responses client, and text/image/audio/file parts for the Chat Completions client.client_prompt_caching.pysample (per review feedback) demonstrating both knobs with the visible cached-token count._prepare_content_for_openaigets an explicittextcase (text previously fell through to theto_dictfallback), and text parts that carry a breakpoint keep list-form content, because the plain-string flattening cannot hold one. The same applies to system/developer messages. Without a breakpoint, the existing string forms are preserved unchanged.What is the impact of these changes?
mode: "explicit"(which disables the implicit breakpoint), a repeated ~1.6k-token prefix reportscached_tokens≈ 3100 on the second call acrossgpt-5.6-luna,gpt-5.6-solandgpt-5.6-terraon the Responses API, and ongpt-5.6-lunavia Chat Completions; the same setup without a breakpoint reports 0. Older models are unaffected; opting in on one surfaces the API's own 400 (prompt_cache_breakpoint is not supported on this model).PromptCacheOptionstypes were introduced in openai 2.45.0, so the import is guarded with a local fallback and the dependency floor stays at 2.25.0; older openai versions keep working (verified against 2.25.0), and sending the option itself requires 2.45.0, as noted in the field docstrings.What do you want reviewers to focus on?
additional_propertiesis the surface you want for the per-part breakpoint, or if you'd rather have a first-class field onContent(happy to rework it).Related Issue
Fixes #7157
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.