Repository navigation
Closed
Labels
Description
Summary
engine.model-routing now works end to end on main: #66234, #66290 and #66291 fixed the
blockers. But the only documentation is the generated frontmatter reference: one sentence plus
the field names. An author can't tell what the values mean, what happens at run time, what's
required, or what doesn't work yet. Before announcing routing, add a guide page that covers the
points below, and link it from the engines reference and the frontmatter reference.
What the guide should cover
- What it does.
- Instead of a fixed
model, the Copilot engine's model and reasoning effort (noneto
max) are chosen per run, before the agent starts. When routing is on,engine.modeland
any effort setting are ignored. - AWF's API proxy classifies the workflow's task text (
user.txt, not gh-aw's system
instructions) with a small LLM call. That text includes anything the workflow interpolates
into its prompt, such as issue bodies; the classifier treats it as untrusted data. - The router, githubnext/gh-aw-router (public),
then ranks the allowed models and efforts using precompiled routing tables.- The tables are fitted on benchmark results and model capability data.
- Allowed models the tables don't cover are never selected.
- Link the router's README for the classification labels, the profiles and its API.
- The choice is advisory: requests for other allowed models are admitted and logged as
deviations.allowed-modelsis the request policy.
- Instead of a fixed
- Fields and values.
goal:cost: cheapest arm that meets the quality bar;cost-speed: also weighs time.
mode:economy,balanced,robust: increasing quality bars, so increasing cost;auto: the classifier recommends one of the three per task.
allowed-models: required. These are the Copilot models routing may choose from, and they
are also the models the run may call.- An example, for instance
goal: cost,mode: auto, and three or four allowed models.
- Requirements.
- Copilot engine only, with the AWF firewall enabled; the compiler rejects anything else.
- AWF versions:
- v0.28.29 or later for routing at all;
- v0.28.33 or later for the sub-agent model split (Separate routing candidates from sub-agent model policy #66234);
- v0.28.35 or later for the routing log.
- Since Route model-routing workflows to version-matched, pinned images #66291 the defaults are AWF v0.28.37 and router
0.1.3, with version-matched,
digest-pinned images, so most authors set nothing. Overridingsandbox.agent.versionneeds
a published AWF release, not just a tag. - Orgs that require approved registries can override the router image with
sandbox.agent.images.router. allowed-modelsmust fit the workflow'smodels.allowedandmodels.blockedpolicy.
GitHub Actions expressions inallowed-modelsare rejected when such a policy is set,
because they can't be checked at compile time.- Models must be available to the organization's Copilot plan; unavailable ones are skipped.
- Routing adds a classifier call per run, typically a fraction of a credit.
- Sub-agents.
- Since Separate routing candidates from sub-agent model policy #66234, inline and imported sub-agents that declare a
model:are admitted, and the
router still only chooses amongallowed-models. - Known constraint: Copilot CLI uses one wire API per session. A sub-agent whose model is
from the other API family than the main model (Claude versus GPT-5) fails with a 400
(gh-aw-firewall#9509). Under routing, that means compared with every allowed model.
- Since Separate routing candidates from sub-agent model policy #66234, inline and imported sub-agents that declare a
- Known limitations, until fixed.
- Threat detection fails in routed workflows (Model routing: threat detection fails in routed workflows, so safe outputs are applied without a threat verdict #66224), but it doesn't fail the workflow.
- Detection defaults to
continue-on-error, so the run still succeeds. - The detection job's failure can be ignored for now. Authors don't need to turn detection
off. - Say plainly that safe outputs are then applied without a threat verdict, so authors who
rely on detection can decide for themselves.
- Detection defaults to
- The selected model can vary between runs of the same task, because the classifier's
recommendation isn't deterministic, especially withmode: auto.
- Threat detection fails in routed workflows (Model routing: threat detection fails in routed workflows, so safe outputs are applied without a threat verdict #66224), but it doesn't fail the workflow.
- Observability.
- The
agentartifact contains
sandbox/firewall/logs/api-proxy-logs/model-routing.jsonl. It holds the classification
(labels, mode), the ranked choices, the selection, the router version, and one record per
request (as_selectedordeviated, with the outcome). token-usage.jsonlhas credits per request, with the classifier call marked
routing_classification.- Mention when
gh aw logsandgh aw auditwill show this.
- The
- Troubleshooting.
- What exit code 78 means (routing failed:
no_route, or the classifier or router
unavailable). - What a degraded classification is (
degraded_reason). - How to read a
deviatedrequest. - The
upstream-errors.jsonlfile.
- What exit code 78 means (routing failed:
Plan
- Add a reference page, for example
docs/src/content/docs/reference/model-routing.md, covering
sections 1–7. - Link it from the engines reference (Copilot section) and from the
model-routingentry in the
frontmatter reference. Expand the schema descriptions forgoal,modeandallowed-models
so the generated reference explains the values. - Mark the feature experimental on the page, and list the known limitations with their issue
links so they can be removed as they're fixed. - Keep the AWF-side details (routing config,
routing.candidateModels, the log schema) as links
to gh-aw-firewall'sdocs/api-proxy-sidecar.mdanddocs/awf-config-spec.mdrather than
duplicating them. Link gh-aw-router for the
router itself: labels, profiles, the/classifyand/routeAPI, and the table format.
Reacted by Copilot
Activity