Model routing: support the Claude, Codex and pi engines on the Copilot API #66714

Problem

engine.model-routing works only with engine.id: copilot. The Claude, Codex and pi engines can already run on the Copilot API (copilot/<model> with permissions: copilot-requests: write), but a workflow that enables routing on any of them fails to compile.

Most of the routing path is already engine-agnostic:

• BuildAWFConfigJSON (pkg/workflow/awf_config_build.go) writes apiProxy.routing, the candidate models and the experimental flag for any engine.
• sandbox_agent_images.go adds the routing images whenever isModelRoutingEnabled is true.
• generateModelRoutingConversationStep runs before every engine's execution step (compiler_yaml_ai_execution.go).
• run_awf_with_startup_retries.sh already treats exit 78 as terminal for any engine when GH_AW_MODEL_ROUTING=1.
• Threat detection already drops ModelRouting for every engine (Keep model routing out of threat detection jobs #66670).

Three things block the other engines:

1. Compile time. validateModelRouting (pkg/workflow/model_routing.go) rejects any engine.id other than copilot.
2. Step env. Only addCopilotWorkflowStepEnv (copilot_engine_execution.go) sets GH_AW_MODEL_ROUTING=1, and only addCopilotModelEnv skips the fixed-model env when routing is on.
3. Runtime. Only copilot_harness.cjs reads routing.selection from AWF /reflect (resolveAWFModelRoutingSelection, applyCopilotRoutingSelection, applyCopilotRoutingArgs) and refuses to start when no valid selection exists. claude_harness.cjs, codex_harness.cjs, pi_models_json.cjs and pi_provider.cjs already fetch /reflect but ignore routing.

Evidence

We simulated pi with routing. For each task, we pinned pi to the model and effort the router had picked, then ran the Copilot engine pinned the same way on the same day. Neither run had apiProxy.routing in awf-config.json. A blind judge scored quality the same for both engines (85 vs 85 out of 100 over five tasks). pi used 37% fewer credits overall and about half on Opus.

The table shows three of those pairs, from gh aw audit (dev build 303b402810):

All six runs succeeded with no errors. Each pair ran the same model at the same effort:

• pi set the effort with engine.config.settings.defaultThinkingLevel, and its session log records thinkingLevel on every message (high ×64, xhigh ×46, xhigh ×34).
• Copilot set the effort with reasoningEffort (high, xhigh, xhigh).

pi chooses the wire API for each model (resolvePiApiForModel in pi_models_json.cjs): Claude on native /v1/messages, GPT on /responses. The Copilot CLI uses a single wire API for each BYOK session (see github/gh-aw-firewall#9558), so it sent Opus over /chat/completions.

pi's savings come from two sources:

• Fewer requests on Opus: 32 vs 61, and 23 vs 33.
• Smaller context per request: about 53–61k cache-read tokens per request vs about 93–96k on Opus, and about 23k vs 40k on GPT-5.6-sol.

pi also accepted thinking level max.

Supporting routing on these engines would let authors combine the router's per-task choice of model and effort with a cheaper engine.

Proposed solution

1. Allow the engines at compile time (pkg/workflow/model_routing.go)

• In validateModelRouting, accept claude, codex and pi alongside copilot, and keep the existing checks for firewall, goal, mode, model policy and AWF version.

• For the non-Copilot engines, require Copilot inference: the engine's ResolveLLMProvider must resolve to the GitHub provider. The router selects only Copilot models, and resolveModelRoutingAllowedModels qualifies every candidate as github-copilot/.

• Add a per-engine compatibility check on allowed-models, called from validateModelRouting:

  • claude: Claude models only, because Claude Code speaks the native Messages API.
  • codex: GPT models served on /responses only.
  • pi: any Copilot model.
  • copilot: unchanged.

  The compiler has no wire-API catalog, so the check would classify models by family name. Candidates that contain expressions cannot be checked at compile time, so the runtime endpoint check in section 3 is the backstop.

• Have warnRoutedSubAgentModels warn when a declared sub-agent model is incompatible with the engine.

• Update the engine.model-routing description in main_workflow_schema.json and the validation error text, which currently say "Copilot" and "supported only by the Copilot engine".

2. Set the routing env and drop fixed-model env for each engine

Factor the routing env into a shared helper used by all four engines, so GH_AW_MODEL_ROUTING=1 is set in one place:

• Claude (claude_engine.go): set the flag in buildClaudeCommandEnv. Skip applyClaudeModelEnvVars when routing is on, which removes ANTHROPIC_MODEL, GH_AW_MODEL_AGENT_CLAUDE and the fallback. That also stops buildClaudeCommandString from appending --model.
• Codex (codex_engine.go): set the flag in buildCodexExecutionEnv. Skip the model env and the fallback expression, and drop the --model expansion in buildCodexCommand.
• pi (pi_engine.go): set the flag in buildPiExecutionEnv, and skip GH_AW_PI_MODEL and GH_AW_PI_NATIVE_PROVIDER. buildPiModelsJSONSetup currently returns nothing unless engine.model is set, and bakes --model aw-gateway/<id> into the command. Under routing, it should always emit the models.json setup without a compile-time model ID or --model.

Fixed-model settings should not override the routed selection. When routing is on, the compiler should warn about engine.model, model variables in engine.env, and engine.config.settings.defaultThinkingLevel, as the schema says it does for Copilot.

3. Apply the selection in each engine's runtime

Move resolveAWFModelRoutingSelection from copilot_harness.cjs into a shared module next to awf_reflect.cjs, and pass in the endpoints each engine accepts. ROUTING_REASONING_EFFORTS from copilot_routing_effort.cjs moves with it.

When GH_AW_MODEL_ROUTING=1, each engine should:

1. Fetch /reflect, whether or not AWF_REFLECT_ENABLED is set.
2. Require routing.status to be selected.
3. Check that wire_model is advertised and that endpoint matches the engine's wire API.
4. Translate the effort to the engine's own setting.
5. Log the same inference routing: mode=awf-routed model=… effort=… line the Copilot harness logs, so audits can read it.

If any step fails, the engine refuses to start, as the Copilot harness does, and never falls back to a default model.

• pi: pi_models_json.cjs already runs inside AWF before the pi CLI and fetches /reflect.
  • Under routing, it reads the selection and writes the models.json entry for wire_model. The API comes from endpoint (/v1/messages → anthropic-messages, /responses → openai-responses, /chat/completions → openai-completions) and is cross-checked with resolvePiApiForModel.
  • --model is currently compiled into the command, so the selected model must reach pi another way. For example, the runtime can write it to settings.json (which pi_runtime.cjs already manages) or to a file the command reads.
  • Set GH_AW_PI_MODEL so pi_provider.cjs and the drivers (pi_agent_core_driver.cjs, pi_rpc_driver.cjs) see the routed model. Otherwise, reject routing in driver mode.
  • Write the effort to settings.defaultThinkingLevel, mapping none to off.
• Claude: in claude_harness.cjs main(), resolve the selection from the /reflect data it already fetches.
  • Require /v1/messages.
  • Set ANTHROPIC_MODEL in buildClaudeChildEnv.
  • Replace any --model in initialArgs, as applyCopilotRoutingArgs does, so normalizeClaudeModelArgs cannot restore a fixed model.
  • Pass the effort with --effort or CLAUDE_CODE_EFFORT_LEVEL, which accept low, medium, high, xhigh and max.
• Codex: codex_harness.cjs main() resolves the model (normalizeCodexModelArgs, injectModelFlagAfterExec) before its best-effort /reflect fetch. Under routing, fetch first.
  • Require /responses.
  • Inject wire_model as the model.
  • Set model_reasoning_effort through a config override or the config.toml that codex_config.cjs writes.

The router's effort set (none, minimal, low, medium, high, xhigh, max) does not map exactly onto every engine. Claude Code has no none or minimal, and Codex has no max. I suggest mapping only where the meaning is preserved, such as pi's none to off, and failing closed otherwise with the effort named in the error, matching how resolveAWFModelRoutingSelection handles an unsupported effort today. Clamping, or giving the router each engine's supported effort set, are alternatives worth discussing.

4. Keep threat detection unrouted

#66670 clears ModelRouting in resetDetectionEngineTaskSettings and cloneThreatDetectionEngineConfig, and both work for any engine. pi's detection normalizes to Copilot. Add per-engine regression tests asserting that the detection job has no GH_AW_MODEL_ROUTING, apiProxy.routing or router images. The tests should also check that inheritedDetectionModel behaves correctly when engine.model is unset under routing.

5. Docs and tests

• Docs: update docs/src/content/docs/reference/model-routing.md:
  • the description and Requirements section, which currently say "Routing requires the Copilot engine";
  • per-engine allowed-models constraints;
  • a per-engine effort mapping table;
  • the routing log line for each engine;
  • the threat-detection limitation, now that Model routing: threat detection fails in routed workflows, so safe outputs are applied without a threat verdict #66224 is closed.
• Go tests:
  • model_routing_test.go: the accepted and rejected engines, incompatible candidates for each engine, and the non-Copilot-provider rejection.
  • Lock-level tests in claude_engine_test.go, codex_engine_test.go and pi_engine_test.go: routing flag present, fixed-model env and --model absent, apiProxy.routing present.
• JS tests:
  • The shared selection module, moved from copilot_harness.test.cjs.
  • claude_harness.test.cjs, codex_harness.test.cjs, pi_models_json.test.cjs and pi_runtime.test.cjs: selection applied; missing, pending or mismatched-endpoint selections fail closed; effort mapping.
• Live validation: rerun t05, t07 and t10 in githubnext/gh-aw-routing-sandbox with routing on for pi, Claude and Codex. Confirm the audit's routing section, the selected model and effort in the engine logs, and the credits.

Out of scope

The Copilot CLI's single wire API per BYOK session (github/gh-aw-firewall#9558).