Repository navigation
Conversation
Reasoning models reject max_tokens on Chat Completions. Reuse the existing model classifier to send max_completion_tokens while keeping other compatible backends on max_tokens. Fixes google#7863
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Problem:
Apigee sends
max_tokenseven when a Chat Completions request targets an OpenAI reasoning model. Those models requiremax_completion_tokens, so settingmax_output_tokensmakes the request fail.Solution:
Use the same reasoning-model check as
OpenAILlmto select the token-limit field after resolving the outbound model ID. Other compatible models keepmax_tokens. An unset limit stays omitted, and zero is forwarded.Testing Plan
Unit Tests:
Added 18 request-payload cases covering reasoning models, classic and non-reasoning chat models, and another compatible backend, with unset, zero and positive limits. Six cases fail on unchanged
491d7dband pass with the fix.Related Apigee, OpenAI integration and import-loading tests: 441 passed. Local type comparison introduced no new errors, and applicable pre-commit checks passed.
Linux pre-commit, full-project type comparisons on Python 3.11–3.13, and the installed-wheel check passed.
Full unit-suite results from tox:
The same 14 failing test IDs and error messages reproduce on unchanged
491d7dbin each resolved environment: the A2A custom-factory test with SDK 1.3 (_httpx_client) and 13 telemetry schema comparisons. The full-suite jobs remain red; all four baseline comparisons passed. Each version also reports 12 expected failures and 2 unexpected passes.Manual End-to-End (E2E) Tests:
Exercised
ApigeeLlm.generate_content_asyncthroughhttpx.MockTransportwith streaming and non-streaming responses. Before the fix, eight of 20 requests send the unsupported parameter. After the fix, all 20 pass. This checks the request and response path without a live model or cloud credentials.The fork run also builds and installs the wheel in a clean Python 3.11 environment with core dependencies only, verifies that the OpenAI SDK is absent, and runs the same 20 requests.
python apigee_token_limit_e2e.py.Checklist
Additional context
The reporter confirmed they are not working on the fix.
Prepared with Codex.