Skip to content

fix(gateway): route v2 openai calls over the responses api - #1166

Merged
gewenyu99 merged 5 commits into
mainfrom
brandon/gateway-responses-shape
Aug 28, 2026
Merged

gewenyu99 merged 5 commits into
mainfrom
brandon/gateway-responses-shape

Conversation

@brandonleung

@brandonleung brandonleung commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Setup-wizard runs against the Go gateway do nothing and still report success. OpenAI refuses function tools combined with reasoning_effort on /v1/chat/completions, and every task in this harness sends both, so each agent turn comes back 400.

Changes

Moves OpenAI-class models onto the Responses API, but only for runs on the new gateway. Anthropic models are untouched.

  • gatewayApiFor now takes the gateway edition. v2 gets openai-responses; legacy stays on chat completions, where litellm already does that routing. The transport change reaches only the runs that need it, not the ~100% still on the old gateway.
  • An empty task queue after seeding fails the run. The harness can return a clean result after every turn was refused, which is how four rejected requests produced a green zero-step run.
  • baseUrl keys off anthropic-messages rather than naming the OpenAI shape, so the /v1 suffix survives the rename.
  • Reasoning effort stays on. OpenAI's error suggests reasoning_effort: 'none', but these calls produce real reasoning today.

Test plan

  • vitest run src/lib/agent: 529 tests pass, including a new case pinning that a v2 run resolves to openai-responses while a legacy run stays on openai-completions.
  • Checked both gateways by hand. Tools plus reasoning_effort return 400 on /v1/chat/completions and succeed on /v1/responses. The legacy gateway serves /v1/responses as well (401 there, against 404 on an unknown path), so the edition gate limits blast radius rather than working around a missing route.

Known gap: the empty-queue guard is not pinned by a test. Nothing drives runOrchestrator end to end today and the neighbouring tests cover pure helpers, so reverting that line fails nothing.

This changes the transport for every OpenAI model on the v2 path, including the scan-triage call, and no unit test touches the wire. A live wizard run with the rollout flag enabled is the real gate.

LLM context

Authored with Claude Opus 5; requires human review.

@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/groq
  • /wizard-ci ai-observability/manual-capture
Show more apps
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

@brandonleung
brandonleung marked this pull request as ready for review August 28, 2026 21:32
@brandonleung
brandonleung requested a review from a team as a code owner August 28, 2026 21:32
@gewenyu99

Copy link
Copy Markdown
Collaborator

/wizard-ci basic-integration/javascript-node

@wizard-ci-bot

wizard-ci-bot Bot commented Aug 28, 2026 •

Copy link
Copy Markdown

🧙 Wizard CI Results

Trigger ID: e2be1ce
Workflow: View run

App Confidence PR YARA
basic-integration/javascript-node/express-todo N/A Failed (logs)
basic-integration/javascript-node/fastify-blog ⏳ pending
basic-integration/javascript-node/hono-links N/A Failed (logs)
basic-integration/javascript-node/koa-notes N/A Failed (logs)
basic-integration/javascript-node/native-http-contacts N/A Failed (logs)

Configuration

Setting Value
Wizard ref brandon/gateway-responses-shape
Context Mill ref main
PostHog ref master

Search for trigger ID e2be1ce in wizard-workbench PRs.

@brandonleung brandonleung changed the title fix(gateway): route openai models over the responses api fix(gateway): route v2 openai calls over the responses api Aug 28, 2026
@gewenyu99

Copy link
Copy Markdown
Collaborator

/wizard-ci basic-integration/javascript-node

@wizard-ci-bot

wizard-ci-bot Bot commented Aug 28, 2026 •

Copy link
Copy Markdown

🧙 Wizard CI Results

Trigger ID: f49bb1e
Workflow: View run

App Confidence PR YARA
basic-integration/javascript-node/express-todo N/A Failed (logs)
basic-integration/javascript-node/fastify-blog N/A Failed (logs)
basic-integration/javascript-node/hono-links N/A Failed (logs)
basic-integration/javascript-node/koa-notes ⏳ pending
basic-integration/javascript-node/native-http-contacts N/A Failed (logs)

Configuration

Setting Value
Wizard ref brandon/gateway-responses-shape
Context Mill ref main
PostHog ref master

Search for trigger ID f49bb1e in wizard-workbench PRs.

brandonleung and others added 5 commits August 28, 2026 18:01
OpenAI rejects function tools combined with reasoning_effort on
/v1/chat/completions, which every task in the pi harness sends.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… empty plan

Legacy keeps chat completions, so the transport change reaches only runs
on the new gateway. An empty task queue after seeding now fails the run
rather than reporting success with zero steps.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…he caller

A 401 means the endpoint does not recognise the credential, which is the
case for the API key CI runs with. The legacy gateway authenticates that
credential itself, so a downgrade grants nothing. 400, 403 and 429 stay
fatal: those are callers the endpoint authenticated and then refused.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The hollow-run abort on the drain covers it, with an error code and a
message for the user.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@brandonleung
brandonleung force-pushed the brandon/gateway-responses-shape branch from bb36963 to bf00bde Compare August 28, 2026 22:04
@brandonleung

Copy link
Copy Markdown
Contributor Author

/wizard-ci basic-integration/javascript-node

@wizard-ci-bot

wizard-ci-bot Bot commented Aug 28, 2026 •

Copy link
Copy Markdown

🧙 Wizard CI Results

Trigger ID: c3e1fec
Workflow: View run

App Confidence PR YARA
basic-integration/javascript-node/express-todo 5/5 #3667 (logs) ✓
basic-integration/javascript-node/fastify-blog 4/5 #3666 (logs) ✓
basic-integration/javascript-node/hono-links 4/5 #3665 (logs) ✓
basic-integration/javascript-node/koa-notes 5/5 #3664 (logs) ✓
basic-integration/javascript-node/native-http-contacts 5/5 #3663 (logs) ✓

Configuration

Setting Value
Wizard ref brandon/gateway-responses-shape
Context Mill ref main
PostHog ref master

Search for trigger ID c3e1fec in wizard-workbench PRs.

@gewenyu99
gewenyu99 merged commit a8c3ef1 into main Aug 28, 2026
17 checks passed
@gewenyu99
gewenyu99 deleted the brandon/gateway-responses-shape branch August 28, 2026 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants