Skip to content

Docs: add a model routing guide for engine.model-routing #66337

Closed

Description

@SivaKesava1

Summary

engine.model-routing now works end to end on main: #66234, #66290 and #66291 fixed the
blockers. But the only documentation is the generated frontmatter reference: one sentence plus
the field names. An author can't tell what the values mean, what happens at run time, what's
required, or what doesn't work yet. Before announcing routing, add a guide page that covers the
points below, and link it from the engines reference and the frontmatter reference.

What the guide should cover

  1. What it does.
    • Instead of a fixed model, the Copilot engine's model and reasoning effort (none to
      max) are chosen per run, before the agent starts. When routing is on, engine.model and
      any effort setting are ignored.
    • AWF's API proxy classifies the workflow's task text (user.txt, not gh-aw's system
      instructions) with a small LLM call. That text includes anything the workflow interpolates
      into its prompt, such as issue bodies; the classifier treats it as untrusted data.
    • The router, githubnext/gh-aw-router (public),
      then ranks the allowed models and efforts using precompiled routing tables.
      • The tables are fitted on benchmark results and model capability data.
      • Allowed models the tables don't cover are never selected.
      • Link the router's README for the classification labels, the profiles and its API.
    • The choice is advisory: requests for other allowed models are admitted and logged as
      deviations. allowed-models is the request policy.
  2. Fields and values.
    • goal:
      • cost: cheapest arm that meets the quality bar;
      • cost-speed: also weighs time.
    • mode:
      • economy, balanced, robust: increasing quality bars, so increasing cost;
      • auto: the classifier recommends one of the three per task.
    • allowed-models: required. These are the Copilot models routing may choose from, and they
      are also the models the run may call.
    • An example, for instance goal: cost, mode: auto, and three or four allowed models.
  3. Requirements.
    • Copilot engine only, with the AWF firewall enabled; the compiler rejects anything else.
    • AWF versions:
    • Since Route model-routing workflows to version-matched, pinned images #66291 the defaults are AWF v0.28.37 and router 0.1.3, with version-matched,
      digest-pinned images, so most authors set nothing. Overriding sandbox.agent.version needs
      a published AWF release, not just a tag.
    • Orgs that require approved registries can override the router image with
      sandbox.agent.images.router.
    • allowed-models must fit the workflow's models.allowed and models.blocked policy.
      GitHub Actions expressions in allowed-models are rejected when such a policy is set,
      because they can't be checked at compile time.
    • Models must be available to the organization's Copilot plan; unavailable ones are skipped.
    • Routing adds a classifier call per run, typically a fraction of a credit.
  4. Sub-agents.
    • Since Separate routing candidates from sub-agent model policy #66234, inline and imported sub-agents that declare a model: are admitted, and the
      router still only chooses among allowed-models.
    • Known constraint: Copilot CLI uses one wire API per session. A sub-agent whose model is
      from the other API family than the main model (Claude versus GPT-5) fails with a 400
      (gh-aw-firewall#9509). Under routing, that means compared with every allowed model.
  5. Known limitations, until fixed.
    • Threat detection fails in routed workflows (Model routing: threat detection fails in routed workflows, so safe outputs are applied without a threat verdict #66224), but it doesn't fail the workflow.
      • Detection defaults to continue-on-error, so the run still succeeds.
      • The detection job's failure can be ignored for now. Authors don't need to turn detection
        off.
      • Say plainly that safe outputs are then applied without a threat verdict, so authors who
        rely on detection can decide for themselves.
    • The selected model can vary between runs of the same task, because the classifier's
      recommendation isn't deterministic, especially with mode: auto.
  6. Observability.
    • The agent artifact contains
      sandbox/firewall/logs/api-proxy-logs/model-routing.jsonl. It holds the classification
      (labels, mode), the ranked choices, the selection, the router version, and one record per
      request (as_selected or deviated, with the outcome).
    • token-usage.jsonl has credits per request, with the classifier call marked
      routing_classification.
    • Mention when gh aw logs and gh aw audit will show this.
  7. Troubleshooting.
    • What exit code 78 means (routing failed: no_route, or the classifier or router
      unavailable).
    • What a degraded classification is (degraded_reason).
    • How to read a deviated request.
    • The upstream-errors.jsonl file.

Plan

  • Add a reference page, for example docs/src/content/docs/reference/model-routing.md, covering
    sections 1–7.
  • Link it from the engines reference (Copilot section) and from the model-routing entry in the
    frontmatter reference. Expand the schema descriptions for goal, mode and allowed-models
    so the generated reference explains the values.
  • Mark the feature experimental on the page, and list the known limitations with their issue
    links so they can be removed as they're fixed.
  • Keep the AWF-side details (routing config, routing.candidateModels, the log schema) as links
    to gh-aw-firewall's docs/api-proxy-sidecar.md and docs/awf-config-spec.md rather than
    duplicating them. Link gh-aw-router for the
    router itself: labels, profiles, the /classify and /route API, and the table format.

Activity

  1. locked and limited conversation to collaborators on Oct 7, 2026
  2. unlocked this conversation on Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions