Description
Title: Model routing: Copilot harness rejects router-selected reasoning efforts none and max
Summary
With engine.model-routing enabled (#64593), the Copilot harness refuses to start Copilot
whenever AWF's routing selection uses reasoning effort none or max:
unexpected error: AWF model routing failed: AWF /reflect returned unsupported reasoning effort max; refusing to start Copilot
Both values are valid end to end:
- gh-aw-router and AWF's routing candidate pool use the effort set
none | minimal | low | medium | high | xhigh | max. - The pinned Copilot CLI (1.0.89) accepts all seven values:
copilot --help→--reasoning-effort <level> [possible values: none, minimal, low, medium, high, xhigh, max].
Only gh-aw's allow-lists are narrower: they accept minimal | low | medium | high | xhigh.
Impact
This affects any routed workflow whose candidates include models that support max or none.
In the published gh-aw-router tables, the top-ranked Copilot choice uses max or none in:
| Profile | Share of label cells |
|---|---|
cost / balanced |
30% |
cost-speed / economy |
22% |
cost / robust |
21% |
cost / economy |
14% |
When this happens, the agent step fails before any work is done. This is not a routing failure
(exit 78). A local AWF run selected gpt-6-astra with effort max, and the Copilot API served
that request successfully.
Analysis
Two allow-lists in actions/setup/js are the only mismatch:
copilot_harness.cjs,resolveAWFModelRoutingSelection(around line 509):if (effort && !["minimal", "low", "medium", "high", "xhigh"].includes(effort)) { return { selection: null, error: `AWF /reflect returned unsupported reasoning effort ${effort}` }; }
copilot_sdk_driver.cjs(around line 113) only forwardsreasoningEffortto the SDK session
whenGH_AW_COPILOT_ROUTING_EFFORTis one ofminimal | low | medium | high | xhigh.
Otherwise it silently drops the effort.
The other code paths already pass the value through unchanged:
applyCopilotRoutingArgs(--reasoning-effort <effort>)applyCopilotRoutingSelection(GH_AW_COPILOT_ROUTING_EFFORT)buildCopilotSDKChildEnv(COPILOT_REASONING_EFFORTfor sub-agent sessions)
Note: @github/copilot-sdk 1.0.13 types SessionConfig.reasoningEffort as
"low" | "medium" | "high" | "xhigh" | "max". The RPC layer uses string, and
copilot_sdk_session.cjs already passes the value as any. The headless server is the same CLI
that accepts all seven values.
Implementation plan
- Add one shared constant in
actions/setup/js(incopilot_harness.cjs, or a small shared
module that the SDK driver also imports):These are the values the pinned Copilot CLI lists forconst ROUTING_REASONING_EFFORTS = Object.freeze(["none", "minimal", "low", "medium", "high", "xhigh", "max"]);
--reasoning-effort. - Use it in
resolveAWFModelRoutingSelectionin place of the inline list. Keep
lower-casing and trimming. Keep the distinction betweeneffort: null(no flag, no env) and
effort: "none"(an explicit value that is passed through). - Use it in
copilot_sdk_driver.cjsfor thereasoningEffortforwarding check, somaxand
nonereachrunWithCopilotSDK. - Add tests:
copilot_harness.test.cjs:resolveAWFModelRoutingSelectionacceptsmaxandnone, and still rejects an unknown
value such as"extreme".applyCopilotRoutingArgswitheffort: "max"produces--reasoning-effort max.effort: "none"produces--reasoning-effort none.effort: nullproduces no flag.applyCopilotRoutingSelectionsetsGH_AW_COPILOT_ROUTING_EFFORTtomaxandnone.
- SDK driver:
reasoningEffortis forwarded formaxandnonewhenGH_AW_MODEL_ROUTING=1.
- Optional hardening: if the Copilot CLI version pin changes, a test or comment should tie
ROUTING_REASONING_EFFORTSto the CLI's documented--reasoning-effortvalues. - Run
make agent-finish.
Verification
- Unit tests above.
- End to end:
- Compile a routed workflow with
engine.model-routing, usingmode: robustorbalancedand
allowed-modelsthat include a model offeringmax(for examplegpt-6-astra). - Confirm the agent step logs
inference routing: mode=awf-routed model=<model> effort=maxand Copilot starts. - Confirm the AWF api-proxy logs a
model_routingrequest event withrouted: "as_selected".
- Compile a routed workflow with
Out of scope
No AWF or gh-aw-router change is needed. Both already use the full effort set.
Activity