Skip to content

fix(apigee): support token limits on reasoning models - #7866

Open
Samecho wants to merge 1 commit into
google:mainfrom
Samecho:codex/apigee-reasoning-token-limit
Open

Samecho wants to merge 1 commit into
google:mainfrom
Samecho:codex/apigee-reasoning-token-limit

Conversation

@Samecho

@Samecho Samecho commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

Problem:

Apigee sends max_tokens even when a Chat Completions request targets an OpenAI reasoning model. Those models require max_completion_tokens, so setting max_output_tokens makes the request fail.

Solution:

Use the same reasoning-model check as OpenAILlm to select the token-limit field after resolving the outbound model ID. Other compatible models keep max_tokens. An unset limit stays omitted, and zero is forwarded.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally. See full-suite results below.

Added 18 request-payload cases covering reasoning models, classic and non-reasoning chat models, and another compatible backend, with unset, zero and positive limits. Six cases fail on unchanged 491d7db and pass with the fix.

Related Apigee, OpenAI integration and import-loading tests: 441 passed. Local type comparison introduced no new errors, and applicable pre-commit checks passed.

Linux pre-commit, full-project type comparisons on Python 3.11–3.13, and the installed-wheel check passed.

Full unit-suite results from tox:

Python Passed Failed Skipped
3.11 22,297 14 97
3.12 22,288 14 98
3.13 22,288 14 98
3.14 22,290 14 96

The same 14 failing test IDs and error messages reproduce on unchanged 491d7db in each resolved environment: the A2A custom-factory test with SDK 1.3 (_httpx_client) and 13 telemetry schema comparisons. The full-suite jobs remain red; all four baseline comparisons passed. Each version also reports 12 expected failures and 2 unexpected passes.

Manual End-to-End (E2E) Tests:

Exercised ApigeeLlm.generate_content_async through httpx.MockTransport with streaming and non-streaming responses. Before the fix, eight of 20 requests send the unsupported parameter. After the fix, all 20 pass. This checks the request and response path without a live model or cloud credentials.

The fork run also builds and installs the wheel in a clean Python 3.11 environment with core dependencies only, verifies that the OpenAI SDK is absent, and runs the same 20 requests.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes. See full-suite results.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules. No new dependency is required.

Additional context

The reporter confirmed they are not working on the fix.

Prepared with Codex.

Reasoning models reject max_tokens on Chat Completions. Reuse the existing
model classifier to send max_completion_tokens while keeping other
compatible backends on max_tokens.

Fixes google#7863
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ApigeeLlm Chat Completions path sends max_tokens to gpt-5 and o-series models

2 participants