Skip to content

Reject unsupported reasoning efforts before proxying - #9520

Merged
lpcox merged 3 commits into
mainfrom
copilot/fix-api-proxy-reasoning-effort
Oct 7, 2026
Merged

lpcox merged 3 commits into
mainfrom
copilot/fix-api-proxy-reasoning-effort

Conversation

Copilot AI commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Requests to gpt-5-mini can include reasoning_effort: "short", which the model rejects. Available logs don’t establish whether the CLI or another component supplied the value.

  • Validate final requests: Check the model and effort after body transforms, including model routing and token steering, against advertised capabilities. Return an actionable 400 for unsupported values; don’t infer a mapping such as short → low.
  • Preserve unknown capabilities: If the proxy lacks authoritative effort metadata, leave the request unchanged.
  • Cover the proxy image: Include the validator in the API proxy Docker image and test chat completions, Responses, and Messages effort fields.
{
  "error": {
    "type": "invalid_request_error",
    "code": "unsupported_reasoning_effort",
    "message": "reasoning_effort \"short\" is not supported by model gpt-5-mini; supported values: [low medium high]"
  }
}

Copilot AI changed the title [WIP] Fix API proxy to reject unsupported reasoning_effort for gpt-5-mini Reject unsupported reasoning efforts before proxying Oct 6, 2026
Copilot AI requested a review from lpcox October 6, 2026 22:49
@lpcox
lpcox marked this pull request as ready for review October 6, 2026 23:31
Copilot AI balanced review requested due to automatic review settings October 6, 2026 23:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Model fallback retries can rewrite the model after validation and proxy an unsupported effort.

Review effort: Balanced
Findings: 1 Medium severity · 1 Low severity

Open (2)
What changed in this PR

Adds reasoning-effort validation based on runtime model capabilities before proxying requests.

Changes:

  • Validates Chat Completions, Responses, and Messages effort fields.
  • Returns actionable 400 errors for unsupported values.
  • Adds unit/integration coverage and packages the validator.
File Description
containers/​api-proxy/​reasoning-effort-validation.js Implements capability validation.
containers/​api-proxy/​reasoning-effort-validation.test.js Tests validation behavior.
containers/​api-proxy/​body-handler.js Runs validation after body transforms.
containers/​api-proxy/​server.token-steering.test.js Covers routed, steered requests.
containers/​api-proxy/​Dockerfile Includes the validator in the image.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread containers/api-proxy/body-handler.js Outdated
}
}

validateReasoningEffort(body, provider, req.url);
Comment on lines +9 to +24
it('rejects an effort not advertised for the resolved model', () => {
replaceRuntimeModels('copilot', [{
id: 'gpt-5-mini',
supportedReasoningEfforts: ['low', 'medium', 'high'],
}]);

expect(() => validateReasoningEffort(
Buffer.from(JSON.stringify({ model: 'gpt-5-mini', reasoning_effort: 'short' })),
'copilot',
'/v1/chat/completions',
)).toThrow(expect.objectContaining({
statusCode: 400,
code: 'unsupported_reasoning_effort',
message: 'reasoning_effort "short" is not supported by model gpt-5-mini; supported values: [low medium high]',
}));
});
@lpcox

lpcox commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator

@copilot address review feedback

Copilot AI commented Oct 6, 2026

Copy link
Copy Markdown
Contributor Author

@copilot address review feedback

Both retry paths now validate the rewritten body at upstream dispatch, and unsupported-effort coverage now includes all three endpoint shapes. Full API-proxy tests pass (113 suites, 2,208 tests). Commit: 13994ac.

@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 92.92% 92.92% ➡️ +0.00%
Statements 91.39% 91.39% ➡️ +0.00%
Functions 89.98% 89.98% ➡️ +0.00%
Branches 85.02% 85.02% ➡️ +0.00%
📁 Per-file Coverage Changes (2 files)
File Lines (Before → After) Statements (Before → After)
src/nvx/one-shot-adapter.ts 83.5% → 82.9% (-0.60%) 80.3% → 79.8% (-0.56%)
src/log-directory-setup.ts 96.8% → 100.0% (+3.18%) 96.9% → 100.0% (+3.13%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

🛡️ Egress verdict from Smoke Copilot Network Isolation

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

💎 Smoke Gemini is crystallizing results on this pull request...

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

Tested by Smoke Chroot

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

🔌 Service connectivity validated by Smoke Services

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ Build Test Suite completed successfully!

Generated by Build Test Suite for #9520

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

🔮 The ancient spirits stir... Smoke Codex awakens to divine this pull request...

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

🔑 Smoke Copilot BYOK AOAI (api-key) is testing Azure OpenAI BYOK (api-key) mode on this pull request...

@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

🪪 Smoke Copilot BYOK AOAI (Entra) is testing Azure OpenAI BYOK (Entra / GitHub OIDC) mode on this pull request...

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ Security Guard completed successfully!

Security review complete: PR #9520 adds reasoning effort validation to the API proxy. No security concerns found. Changes implement input validation that prevents unsupported reasoning efforts from being proxied upstream, with proper error handling (400 responses). No weakened firewall rules, expanded egress, capability additions, or security regressions detected. PR passes security review.

Generated by Security Guard for #9520

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Smoke Cloud Hypervisor completed. Cloud Hypervisor + Copilot passed.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Claude passed

Generated by Smoke Claude for #9520

@lpcox
lpcox enabled auto-merge (squash) October 6, 2026 23:58
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Cloud Hypervisor + Copilot Smoke Test

  1. PASS — list_pull_requests call succeeded (returned PR data for github/gh-aw-firewall).
  2. PASS — https://github.com returned HTTP 200.
  3. PASS — Wrote and read back unique line in /tmp/gh-aw/agent/smoke-cloud-hypervisor-${GITHUB_RUN_ID}.txt.
  4. PASS — (example.com/redacted) was blocked (curl exit 60, HTTP code 000), confirming AWF domain allowlist still enforced despite CLI URL approvals.

All checks passed.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Smoke services: ✅ Redis PONG · ✅ pg_isready accepting connections · ✅ SELECT 1 = 1. PASS

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Smoke Claude Results

  • API status: ✅ PASS
  • GH check: ✅ PASS
  • File status: ✅ PASS

Overall result: PASS

Generated by Smoke Claude for #9520 · claude · haiku45 · 45.4 AIC · ⊞ 6.1K · ◷
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Copilot BYOK (Direct) Smoke Test: PASS ✅

Test Results:

Running in direct BYOK mode via api-proxy sidecar.

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

OTEL smoke test (re-run locally)

  • ✅ S1: otel.js loads and exports its functions.
  • ✅ S2: 3 suites, 68 tests passed.
  • ✅ S3: Trace context is forwarded in env-passthrough.ts and the OTEL vars in api-proxy-env-config.ts.
  • ✅ S4: onUsage hook is present in token-tracker-http.js.
  • ⚠️ S5: The "Collect OTEL diagnostics" post-step hasn't run, so span export is unverified.

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions github-actions Bot added the smoke-copilot-network-isolation Copilot network-isolation egress smoke test label Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

EGRESS_RESULT allow=pass deny=pass

  • ✅ Allowed: api.github.com returned HTTP 403, curl exit 0
  • ✅ Blocked: example.com failed (self-signed cert error, so a TLS-interception proxy denied it)
    Overall: PASS (@lpcox)

🛡️ Egress verdict from Smoke Copilot Network Isolation
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot: PASS ✅

  • MCP (PR read: "Reject unsupported reasoning efforts before proxying"): ✅
  • github.com HTTP 200: ✅
  • File write/read: ✅
    Author: @Copilot · Assignees: @lpcox @Copilot

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia ❌ not run ❌ FAIL
Bun hono ❌ not run ❌ FAIL
C++ fmt ✅ N/A ✅ PASS
C++ json ✅ N/A ✅ PASS
Deno oak N/A not run ❌ FAIL
Deno std N/A not run ❌ FAIL
.NET hello-world ✅ N/A ✅ PASS
.NET json-parse ✅ N/A ✅ PASS
Go color ✅ 1/1 passed ✅ PASS
Go env ✅ 1/1 passed ✅ PASS
Go uuid ✅ 1/1 passed ✅ PASS
Java gson ✅ 1/1 passed ✅ PASS
Java caffeine ✅ 1/1 passed ✅ PASS
Node.js clsx ✅ passed ✅ PASS
Node.js execa ✅ passed ✅ PASS
Node.js p-limit ✅ passed ✅ PASS
Rust fd ✅ 1/1 passed ✅ PASS
Rust zoxide ✅ 1/1 passed ✅ PASS

Overall: 6/8 ecosystems passed — FAIL

Failures

  • Bun / Deno: the curl ... | bash / | sh installer commands were denied by the sandbox permission policy ("Permission denied and could not request permission"). Neither runtime is preinstalled, so the tests were not run.
  • Java note: the default ~/.m2/repository could not be created (LocalRepositoryNotAccessibleException). Both projects passed after I set -Dmaven.repo.local to a temp directory.

All 8 repos cloned successfully. The build-test label was not added because of the failures.

Generated by Build Test Suite for #9520 · copilot · auto · 27.1 AIC · ⊞ 12K · ◷
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Chroot Version Comparison

Runtime Host Version Chroot Version Match?
Python Python 3.12.14 Python 3.12.14 ✅
Node.js v24.21.0 v22.23.2 ❌
Go go1.22.12 go1.22.12 ✅

Result: Not all tests passed. Node.js differs between host and chroot, so the smoke-chroot label was not added.

Tested by Smoke Chroot
Add label ready-for-aw to run again

This branch had an error being deployed

1 failed deployment
aoai-model — 13994ac3 Deployed Oct 7, 2026 by lpcox via conclusion #1895
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

API proxy path allows unsupported reasoning_effort "short" for gpt-5-mini

3 participants