Use catalogue-supported efforts in model routing smoke by Copilot · Pull Request #9929 · github/gh-aw-firewall

Copilot · 2026-10-13T18:05:48Z

The effort-routed Copilot smoke step requests max, which gpt-5.4-mini does not support, causing exit 78 even after routed inference succeeds.

Catalogue-driven requests: Find the selected wire model in /reflect and exercise its lowest and highest supported efforts using the fixed ranking none → minimal → low → medium → high → xhigh → max, regardless of catalogue order.
Explicit validation: Require nonempty effort metadata, reject unrecognized values, and assert that the routed effort is supported before inference.
Scope: Reuse the routed request body for explicit-effort requests. Leave routing-failure accounting and other workflow steps unchanged.

For the current catalogue, the explicit requests become none and xhigh, rather than none and max.

Fixes Model routing smoke: effort step asks gpt-5.4-mini for 'max', which it does not support (still red after #9869) #9927

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot

🟢 Approval recommended

The focused change satisfies the issue requirements without altering routing behavior.

0 open findings

What changed in this PR

Updates the model-routing smoke test to use catalogue-supported reasoning efforts.

Changes:

Validates routed effort metadata against /reflect.
Selects lowest and highest supported efforts using a fixed ranking.
Reuses the routed request body for explicit-effort checks.

File	Description
`.github/workflows/test-model-routing.yml`	Adds catalogue-driven effort validation and requests.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

github-actions · 2026-10-13T18:18:20Z

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

github-actions · 2026-10-13T18:24:03Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	93.16%	93.17%	📈 +0.01%
Statements	91.71%	91.72%	📈 +0.01%
Functions	90.38%	90.38%	➡️ +0.00%
Branches	85.47%	85.48%	📈 +0.01%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/log-directory-setup.ts`	96.8% → 100.0% (+3.18%)	96.9% → 100.0% (+3.13%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

github-actions · 2026-10-13T18:37:58Z

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

🔑 BYOK report filed by Smoke Copilot BYOK

github-actions · 2026-10-13T18:37:58Z

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

📡 OTel tracing validated by Smoke OTel Tracing

github-actions · 2026-10-13T18:37:58Z

🔌 Smoke Services — All services reachable! ✅

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-10-13T18:37:59Z

🪪 Smoke Copilot BYOK AOAI (Entra) is testing Azure OpenAI BYOK (Entra / GitHub OIDC) mode on this pull request...

github-actions · 2026-10-13T18:37:59Z

✅ Smoke Claude passed

Generated by Smoke Claude for #9929

github-actions · 2026-10-13T18:38:01Z

🔑 Smoke Copilot BYOK AOAI (api-key) is testing Azure OpenAI BYOK (api-key) mode on this pull request...

github-actions · 2026-10-13T18:38:01Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

Smoke Codex

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Smoke Codex · gpt54 · 12.3 AIC · ⊞ 20.9K · ◷

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-10-13T18:38:04Z

🚀 Security Guard has started processing this pull request

github-actions · 2026-10-13T18:38:04Z

💎 Smoke Gemini is crystallizing results on this pull request...

github-actions · 2026-10-13T18:38:04Z

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

Tested by Smoke Chroot

github-actions · 2026-10-13T18:38:04Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Copilot

github-actions · 2026-10-13T18:38:27Z

Smoke Cloud Hypervisor completed. Cloud Hypervisor + Copilot passed.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor

github-actions · 2026-10-13T18:38:29Z

✅ Build Test Suite completed successfully!

Generated by Build Test Suite for #9929

github-actions · 2026-10-13T18:39:23Z

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

🛡️ Egress verdict from Smoke Copilot Network Isolation

github-actions · 2026-10-13T18:47:47Z

Smoke Test: Claude Engine Validation

Check	Status
API	✅ PASS
gh CLI	✅ PASS
File	✅ PASS

Overall result: PASS

Generated by Smoke Claude for #9929 · claude · haiku45 · 48.9 AIC · ⊞ 6.1K · ◷
_{Add label ready-for-aw to run again}

github-actions · 2026-10-13T18:48:08Z

Smoke Copilot: PASS ✅

GitHub MCP: ✅ (latest merged: "feat: add staged Phase 4 pilot tiers")
github.com HTTP 200: ✅
File write/read: ✅
PR: "Use catalogue-supported efforts in model routing smoke" — @Copilot, assignees @lpcox @Copilot

📰 BREAKING: Report filed by Smoke Copilot
_{Add label ready-for-aw to run again}

github-actions · 2026-10-13T18:48:14Z

Smoke Test: Copilot BYOK Results

✅ GitHub MCP Connectivity - Verified via list_pull_requests
✅ GitHub.com Connectivity - HTTP 200
✅ File Write/Read Test - /tmp/gh-aw/agent/smoke-test-copilot-byok.txt exists
✅ BYOK Inference - Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy → api.githubcopilot.com

Status: PASS — All tests passed. @Copilot @lpcox

🔑 BYOK report filed by Smoke Copilot BYOK
_{Add label ready-for-aw to run again}

github-actions · 2026-10-13T18:48:17Z

Cloud Hypervisor + Copilot Smoke Test

List PRs (github/gh-aw-firewall): PASS — PR feat: add staged Phase 4 pilot tiers #9931 (closed)
curl https://github.com: PASS — HTTP 200
Write/read temp file: PASS — wrote and read back smoke-cloud-hypervisor-38447865292.txt
curl (example.com/redacted) (should be blocked): PASS — HTTP 000 (connection blocked by firewall)

Overall: PASS ✅

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor
_{Add label ready-for-aw to run again}

github-actions · 2026-10-13T18:48:41Z

OTEL smoke test

✅ S1: otel.js loads and exports its functions
✅ S2: 3 OTEL test suites, 68/68 tests pass
✅ S3: env forwarding present in env-passthrough.ts (trace/parent span IDs) and api-proxy-env-config.ts (OTEL vars)
✅ S4: onUsage hook present in token-tracker-http.js
✅ S5: /tmp/gh-aw/otel.jsonl exists with 1 line (spans were recorded)

📡 OTel tracing validated by Smoke OTel Tracing
_{Add label ready-for-aw to run again}

github-actions · 2026-10-13T18:48:43Z

Smoke services: ✅ Redis PONG, ✅ pg_isready, ✅ SELECT 1. Overall: PASS

🔌 Service connectivity validated by Smoke Services
_{Add label ready-for-aw to run again}

github-actions · 2026-10-13T18:49:52Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	pass	✅ PASS
Go	env	✅	pass	✅ PASS
Go	uuid	✅	pass	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	pass	✅ PASS
Node.js	execa	✅	pass	✅ PASS
Node.js	p-limit	✅	pass	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — PASS

Notes: Bun and Deno were installed via npm (curl installers were blocked). Maven used -Dmaven.repo.local because ~/.m2 was root-owned and non-writable; the pre-existing settings.xml was used.

Generated by Build Test Suite for #9929 · copilot · auto · 24.5 AIC · ⊞ 11.9K · ◷
_{Add label ready-for-aw to run again}

github-actions · 2026-10-13T18:50:07Z

Smoke Test

feat: add staged Phase 4 pilot tiers ✅
fix: scope smoke workflow token to host steps ✅
PR detail lookup ✅
Playwright title check ❌
File write + cat ✅
Build AWF ✅
Overall: FAIL

🔮 The oracle has spoken through Smoke Codex
_{Add label ready-for-aw to run again}

github-actions · 2026-10-13T18:50:08Z

EGRESS_RESULT allow=pass deny=pass

✅ Allowed domain (api.github.com → 200)
✅ Blocked domain (example.com blocked; curl TLS cert error)

Overall: PASS — cc @lpcox

🛡️ Egress verdict from Smoke Copilot Network Isolation
_{Add label ready-for-aw to run again}

github-actions · 2026-10-13T18:50:40Z

Chroot Version Comparison

Runtime	Host Version	Chroot Version	Match?
Python	Python 3.12.15	Python 3.12.15	✅
Node.js	v24.21.0	v22.23.2	❌
Go	go1.22.12	go1.22.12	✅

Result: Not all tests passed. Node.js differs between host and chroot, so the smoke-chroot label was not added.

Tested by Smoke Chroot
_{Add label ready-for-aw to run again}

Initial plan

f736dff

Copilot AI assigned Copilot and lpcox Oct 13, 2026

Copilot AI linked an issue Oct 13, 2026 that may be closed by this pull request

Model routing smoke: effort step asks gpt-5.4-mini for 'max', which it does not support (still red after #9869) #9927

Closed

Copilot started work on behalf of lpcox October 13, 2026 18:06

View session

fix: use supported catalogue efforts in routing smoke

6207977

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix model routing smoke test for gpt-5.4-mini~~ Use catalogue-supported efforts in model routing smoke Oct 13, 2026

Copilot finished work on behalf of lpcox October 13, 2026 18:14

Copilot AI requested a review from lpcox October 13, 2026 18:14

lpcox marked this pull request as ready for review October 13, 2026 18:15

Copilot AI balanced review requested due to automatic review settings October 13, 2026 18:15

Copilot started reviewing on behalf of lpcox October 13, 2026 18:16

View session

Copilot AI reviewed Oct 13, 2026

View reviewed changes

lpcox added the ready-for-aw label Oct 13, 2026

lpcox enabled auto-merge (squash) October 13, 2026 18:37

lpcox had a problem deploying to aoai-model October 13, 2026 18:38 — with GitHub Actions Failure

github-actions Bot added smoke-claude smoke-copilot labels Oct 13, 2026

github-actions Bot added the smoke-copilot-byok label Oct 13, 2026

github-actions Bot added the smoke-cloud-hypervisor label Oct 13, 2026

lpcox temporarily deployed to aoai-model October 13, 2026 18:48 — with GitHub Actions Inactive

lpcox had a problem deploying to aoai-model October 13, 2026 18:48 — with GitHub Actions Failure

github-actions Bot added the smoke-services label Oct 13, 2026

github-actions Bot added the build-test label Oct 13, 2026

github-actions Bot added the smoke-copilot-network-isolation Copilot network-isolation egress smoke test label Oct 13, 2026

lpcox merged commit 7f4653f into main Oct 13, 2026
129 of 139 checks passed

lpcox deleted the copilot/fix-model-routing-smoke-test branch October 13, 2026 18:58

Conversation

Copilot AI commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026

✅ Coverage Check Passed

Overall Coverage

Uh oh!

github-actions Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Smoke Codex

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 13, 2026

Smoke Test: Claude Engine Validation

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026

Smoke Test: Copilot BYOK Results

Uh oh!

github-actions Bot commented Oct 13, 2026

Cloud Hypervisor + Copilot Smoke Test

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026

🏗️ Build Test Suite Results

Uh oh!

github-actions Bot commented Oct 13, 2026

Smoke Test

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026

Chroot Version Comparison

Uh oh!

Uh oh!

Copilot AI commented Oct 13, 2026 •

edited

Loading

github-actions Bot commented Oct 13, 2026 •

edited

Loading

github-actions Bot commented Oct 13, 2026 •

edited

Loading

github-actions Bot commented Oct 13, 2026 •

edited

Loading

github-actions Bot commented Oct 13, 2026 •

edited

Loading

github-actions Bot commented Oct 13, 2026 •

edited

Loading

github-actions Bot commented Oct 13, 2026 •

edited

Loading

github-actions Bot commented Oct 13, 2026 •

edited

Loading

github-actions Bot commented Oct 13, 2026 •

edited

Loading

github-actions Bot commented Oct 13, 2026 •

edited

Loading

github-actions Bot commented Oct 13, 2026 •

edited

Loading