Repository navigation
Copilot auto resolves to gpt-5.6-luna-utility, which rejects the agent's custom tools (and detection alias no longer resolves) #72353
Description
We run a Copilot-engine workflow (compiled with v0.89.21, no explicit model) in a private repo. It started failing on 2026-10-27 without any change on our side. Since the repo is private, I've inlined the relevant log lines instead of linking runs.
auto picks gpt-5.6-luna-utility
Last successful run before the failure (2026-10-20):
[copilot-harness] copilot model alias resolution: 'auto' -> 'gpt-5.6-luna'
Failing run (2026-10-27), same compiler version and same workflow source:
[copilot-harness] copilot model alias resolution: 'auto' -> 'gpt-5.6-luna-utility'
[copilot-harness] inference routing: mode=cli configuredModel="gpt-5.6-luna-utility" endpoint=managed-by-copilot-cli
400 Unsupported Responses custom tool '(unknown)'. Only 'apply_patch' is supported through the Copilot compatibility adapter.
All four harness retries fail the same way within about a second. The harness classifies it as failureClass=partial_execution with isModelNotSupportedError=false, so the retries can't succeed.
From what I can see in /models, gpt-5.6-luna-utility has model_picker_enabled: false and no policy state, while gpt-5.6-luna is picker-enabled and enabled for our org. My understanding is that the utility model isn't meant to be a general agent model, so I think auto shouldn't resolve to it. It showed up in the 2026-10-26 model inventory (#72194) as "Covered by gpt-5.6 alias", which I suspect is how it became eligible.
Pinning GH_AW_MODEL_AGENT_COPILOT=gpt-5.6-luna fixed it for us.
detection alias no longer resolves
Separately, threat detection has been failing for us since around 2026-10-20:
[engine] [copilot-harness] copilot model alias resolution: known alias 'detection' did not resolve against catalog
[engine] [copilot-harness] copilot model alias resolution failed: model-catalog retrieval prevented alias resolution for 'detection' after a bounded refresh — refusing to start Copilot with an unresolved alias
Before that it resolved to gpt-5.4-mini, which the inventory reported as model_unavailable on 2026-10-19/20 (#70675). Detection runs with continue-on-error, so the safe outputs still go through, but detection itself silently stops doing its job. Recompiling with a version that includes #68818 lets us override the model via GH_AW_MODEL_DETECTION_COPILOT, but I think the default alias should probably keep pointing at something available.
I might be missing context on how these aliases are meant to be maintained, so please don't hesitate to correct me if this is already tracked somewhere.
Questions
- Should
autoexclude models that aren't picker-enabled, or aren't enabled by org policy? - Is the
detectionalias supposed to follow model retirements automatically, or are workflows expected to pin a detection model? - Could the harness treat
400 Unsupported Responses custom toolas a model-not-supported failure? That would at least surface a clearer error thanpartial_execution.
Activity
danielmarbach commentedon Oct 27, 2026
A follow-up after a few more hours of poking at this. Pinning
gpt-5.6-lunaonly helped as long as the workflow was compiled with v0.89.21. After recompiling with v0.93.5 (and in two other repos already on v0.93.2), the same model fails differently:Same Copilot CLI (1.0.101) in both cases. The difference I can see is the AWF stack:
api-proxy:0.28.23works,api-proxy:0.28.70fails. I couldn't find whereprompt_cache_breakpointgets added, so I might be pointing at the wrong component. The harness again classifies it aspartial_execution, so all four retries fail the same way.What we tried on v0.93.x, all pinned via
GH_AW_MODEL_AGENT_COPILOT:auto→gpt-5.6-luna-utility400 Model "gpt-5.6-luna-utility" has no AI credits pricing and no default pricing is configuredgpt-5.6-luna400 prompt_cache_breakpoint is not supported on this modelgpt-6-luna400 The requested model is not supported.claude-sonnet-5.5400 The requested model is not supported.gpt-5.3-codexI'm not sure whether those two rejections come from our org's Copilot policy or from the agentic-workflows integration catalog, as the v0.93.5 resolution error hints. The model list I compared against came from a personal Copilot plan, so it wasn't a fair comparison. Either way, it would help a lot if the set of models a workflow can actually use were visible somewhere before a run fails.
The same applies to my original post: the
model_picker_enabledand policy details I quoted forgpt-5.6-lunaandgpt-5.6-luna-utilitycame from that personal plan, not from our org. Sorry for the noise.For anyone else landing here: setting both
GH_AW_MODEL_AGENT_COPILOTandGH_AW_MODEL_DETECTION_COPILOTtogpt-5.3-codexon a v0.93.5 lock got us running again, including threat detection.danielmarbach commentedon Oct 27, 2026
Thanks! When will this roll out?