Skip to content

fix(api-proxy): accept fenced classifier JSON replies - #9930

Merged
lpcox merged 2 commits into
mainfrom
copilot/fix-haiku-classifier-output
Oct 13, 2026
Merged

lpcox merged 2 commits into
mainfrom
copilot/fix-haiku-classifier-output

Conversation

Copilot AI commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Haiku wraps valid classifier JSON in Markdown fences, causing Haiku-only pools to route without classification and record invalid_classifier_output.

  • Normalization: Strip one complete wrapping fence, optionally labelled json, with surrounding whitespace. Apply the existing JSON parser and schema validation to its contents.
  • Strictness: Continue rejecting surrounding prose, multiple fenced blocks, other labels, malformed JSON, and invalid schemas. Extraction, retry behavior, and degraded reasons remain unchanged.
  • Regression coverage: Add named contract tests and controller coverage for classified Haiku selections and still-invalid fenced replies.

The validator would accept:

```json
{"labels":{"task_type":"explain","scope":"local","task_complexity":"unknown"},"mode":"economy"}
```

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix classifier output rejection due to Markdown wrapping fix(api-proxy): accept fenced classifier JSON replies Oct 13, 2026
Copilot AI requested a review from lpcox October 13, 2026 18:13
@lpcox
lpcox marked this pull request as ready for review October 13, 2026 18:14
Copilot AI balanced review requested due to automatic review settings October 13, 2026 18:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused implementation satisfies the stated strictness requirements with comprehensive regression coverage.

0 open findings

What changed in this PR

Adds strict normalization for Haiku classifier responses wrapped in Markdown fences while preserving existing schema validation and degraded-routing behavior.

Changes:

  • Unwraps one complete json or unlabelled code fence.
  • Rejects prose, multiple blocks, unsupported labels, malformed JSON, and invalid schemas.
  • Adds contract and controller regression coverage.
File Description
containers/​api-proxy/​routing-contract.js Normalizes fenced classifier output before parsing.
containers/​api-proxy/​routing-contract.test.js Covers accepted and rejected fence variants.
containers/​api-proxy/​routing-controller.test.js Verifies classification and degradation records.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 93.16% 93.17% 📈 +0.01%
Statements 91.71% 91.72% 📈 +0.01%
Functions 90.38% 90.38% ➡️ +0.00%
Branches 85.47% 85.48% 📈 +0.01%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/log-directory-setup.ts 96.8% → 100.0% (+3.18%) 96.9% → 100.0% (+3.13%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@lpcox
lpcox enabled auto-merge (squash) October 13, 2026 18:38
@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

🔮 The oracle has spoken through Smoke Codex

@github-actions

Copy link
Copy Markdown
Contributor

🪪 Smoke Copilot BYOK AOAI (Entra) is testing Azure OpenAI BYOK (Entra / GitHub OIDC) mode on this pull request...

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

✅ Build Test Suite completed successfully!

Generated by Build Test Suite for #9930

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Claude passed

Generated by Smoke Claude for #9930

@github-actions

Copy link
Copy Markdown
Contributor

💎 Smoke Gemini is crystallizing results on this pull request...

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Smoke Cloud Hypervisor completed. Cloud Hypervisor + Copilot passed.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

Tested by Smoke Chroot

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

🛡️ Egress verdict from Smoke Copilot Network Isolation

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

🔌 Service connectivity validated by Smoke Services

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions

Copy link
Copy Markdown
Contributor

🔑 Smoke Copilot BYOK AOAI (api-key) is testing Azure OpenAI BYOK (api-key) mode on this pull request...

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

✅ Security Guard completed successfully!

Security Guard

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Security Guard · haiku45 · 4.06 AIC · ⊞ 6.0K · ◷

Generated by Security Guard for #9930

@github-actions github-actions Bot added the smoke-copilot-network-isolation Copilot network-isolation egress smoke test label Oct 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

EGRESS_RESULT allow=pass deny=pass

  • ✅ Allowed domain (api.github.com): HTTP 200
  • ✅ Blocked domain (example.com): blocked (curl failed with SSL cert error, no response returned)

Overall: PASS. cc @lpcox

🛡️ Egress verdict from Smoke Copilot Network Isolation
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Cloud Hypervisor + Copilot

  1. PASS — list_pull_requests returned closed PR feat: add staged Phase 4 pilot tiers #9931
  2. PASS — https://github.com returned 200
  3. PASS — wrote/read unique line in /tmp/gh-aw/agent/smoke-cloud-hypervisor-38448333600.txt
  4. PASS — (example.com/redacted) blocked with 000 (connection refused by firewall)

All checks passed.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Claude Engine Validation

Check Status
API ✅ PASS
gh CLI ✅ PASS
File ✅ PASS

Overall result: PASS

Generated by Smoke Claude for #9930 · claude · haiku45 · 24.1 AIC · ⊞ 6.1K · ◷
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct Mode)

✅ All tests passed — Direct BYOK mode active (COPILOT_PROVIDER_API_KEY → api-proxy → api.githubcopilot.com)

  • ✅ MCP: GitHub connectivity confirmed
  • ✅ HTTP: Code 200 (github.com accessible)
  • ✅ File: Smoke test file written and readable
  • ✅ BYOK Inference: Direct mode working (Copilot responding via api-proxy sidecar)

Status: PASS
@lpcox

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Copilot: PASS

  • PR: "fix(api-proxy): accept fenced classifier JSON replies"
  • ✅ GitHub MCP
  • ✅ github.com (HTTP 200)
  • ✅ File write/read
    @Copilot (author), assignees: @lpcox @Copilot

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

OTEL smoke test

  • ✅ S1 Module loading: otel.js exports startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled (+ internals)
  • ✅ S2 Tests: 68/68 passed (3 suites)
  • ✅ S3 Env forwarding: trace context in env-passthrough.ts; OTEL vars in api-proxy-env-config.ts
  • ✅ S4 Token tracker: onUsage hook present in token-tracker-http.js
  • ✅ S5 Diagnostics: /tmp/gh-aw/otel.jsonl has 1 record

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Services smoke test: ✅ Redis PONG, ✅ pg_isready accepting connections, ✅ SELECT 1 = 1. PASS

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Chroot Version Comparison

Runtime Host Version Chroot Version Match?
Python Python 3.12.15 Python 3.12.15 ✅ YES
Node.js v24.21.0 v22.23.2 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

Result: Not all tests passed (Node.js version mismatch), so the smoke-chroot label was not added.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia ❌ not run ❌ FAIL
Bun hono ❌ not run ❌ FAIL
C++ fmt ✅ N/A ✅ PASS
C++ json ✅ N/A ✅ PASS
Deno oak N/A not run ❌ FAIL
Deno std N/A not run ❌ FAIL
.NET hello-world ✅ N/A ✅ PASS
.NET json-parse ✅ N/A ✅ PASS
Go color ✅ pass ✅ PASS
Go env ✅ pass ✅ PASS
Go uuid ✅ pass ✅ PASS
Java gson ✅ 1/1 passed ✅ PASS
Java caffeine ✅ 1/1 passed ✅ PASS
Node.js clsx ✅ passed ✅ PASS
Node.js execa ✅ passed ✅ PASS
Node.js p-limit ✅ passed ✅ PASS
Rust fd ✅ 1/1 passed ✅ PASS
Rust zoxide ✅ 1/1 passed ✅ PASS

Overall: 6/8 ecosystems passed — FAIL

Failures

  • Bun and Deno: the curl | bash / curl | sh installers were blocked by the sandbox (permission denied). Bun and Deno are not preinstalled, so no tests ran.
  • Java note: ~/.m2 is not writable (LocalRepositoryNotAccessibleException). Java passed using -Dmaven.repo.local=/tmp/gh-aw/agent/m2.

Generated by Build Test Suite for #9930 · copilot · auto · 17 AIC · ⊞ 11.9K · ◷
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

feat: add staged Phase 4 pilot tiers
Fix Codex tool-output IDs in Copilot Responses translation
Merged PR review ✅
PR detail lookup ✅
Playwright title check ❌
File write/read ✅
Discussion lookup/comment ✅
Build AWF ✅
Overall status: FAIL

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@lpcox
lpcox merged commit c273a58 into main Oct 13, 2026
133 of 139 checks passed
@lpcox
lpcox deleted the copilot/fix-haiku-classifier-output branch October 13, 2026 18:58

This branch had an error being deployed

1 failed deployment
aoai-model — cf69fca8 Deployed Oct 13, 2026 by lpcox via conclusion #1951
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Routing]: Classifier replies wrapped in a Markdown code fence are rejected, so Haiku-only pools always route unclassified

3 participants