Repository navigation
Replace Docker SBX CI smoke tests with Cloud Hypervisor - #8838
Conversation
Retire the deprecated Docker SBX smoke workflows. Promote Cloud Hypervisor smoke coverage to the ready-for-aw CI gate. Remove obsolete post-processing and tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Documentation PreviewDocumentation has been built for this PR. To view locally:
Built from commit 528d362 |
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (2 files)
Coverage comparison generated by |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The new mandatory gate has a known hosted-runner preflight failure, and supported SBX runtime coverage is removed.
Get a fresh assessment by requesting another Copilot review.
Review effort: Balanced
Findings: 1
Open (2)
What changed in this PR
Replaces deprecated Docker SBX CI smoke coverage with Cloud Hypervisor and makes it the standard agentic CI gate.
Changes:
- Removes Docker SBX smoke workflows and generated artifacts.
- Promotes Cloud Hypervisor to the
ready-for-awgate. - Cleans up obsolete post-processing, tests, and documentation.
| File | Description |
|---|---|
| scripts/ci/smoke-docker-sbx-workflow.test.ts | Removes obsolete SBX workflow tests. |
| scripts/ci/ready-for-aw-workflows.test.ts | Adds Cloud Hypervisor to required CI workflows. |
| scripts/ci/postprocess-smoke-workflows.ts | Removes Docker SBX post-processing. |
| docs/sbx-integration.md | Documents retired SBX smoke workflows. |
| .github/workflows/smoke-playwright-docker-sbx.md | Removes the SBX Playwright workflow. |
| .github/workflows/smoke-playwright-docker-sbx.lock.yml | Removes its generated workflow. |
| .github/workflows/smoke-docker-sbx.md | Removes the primary SBX smoke workflow. |
| .github/workflows/smoke-docker-sbx-codex.md | Removes SBX Codex coverage. |
| .github/workflows/smoke-docker-sbx-claude.md | Removes SBX Claude coverage. |
| .github/workflows/smoke-docker-sbx-build-test.md | Removes SBX build/test coverage. |
| .github/workflows/smoke-cloud-hypervisor.md | Changes the trigger to ready-for-aw. |
| .github/workflows/smoke-cloud-hypervisor.lock.yml | Regenerates the promoted workflow. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| workflow_dispatch: | ||
| label_command: | ||
| name: test-cloud-hypervisor | ||
| name: ready-for-aw |
| - gh-aw workflow frontmatter: `sandbox.agent.runtime: docker-sbx`. The | ||
| Docker SBX CI smoke workflows have been retired in favor of their Cloud | ||
| Hypervisor equivalents. |
|
@copilot address review feedback |
|
❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...
|
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤
|
|
❌ Smoke Gemini reports failed. Facets need polishing...
|
|
🔌 Smoke Services — All services reachable! ✅
|
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "msfeed25.pkgs.visualstudio.com"See Network Configuration for more information.
|
|
✅ Build Test Suite completed successfully!
|
|
🚀 Security Guard has started processing this pull request |
|
🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅ Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "example.com"See Network Configuration for more information.
|
|
✅ Smoke Claude passed Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.anthropic.com"See Network Configuration for more information.
|
|
❌ Smoke Copilot BYOK reports failed to deliver outputs. BYOK mode investigation needed...
|
|
Smoke Cloud Hypervisor reports failed. Cloud Hypervisor + Copilot failed.
|
|
Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.
|
|
📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅
|
|
❌ Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed...
|
Addressed in |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
github-actions
Bot
added
the
smoke-copilot-network-isolation
|
Smoke Cloud Hypervisor reports failed. Cloud Hypervisor + Copilot failed.
|
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓
|
|
❌ Smoke Gemini reports failed. Facets need polishing...
|
|
📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅
|
|
🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅ Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "example.com"See Network Configuration for more information.
|
|
Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.
|
|
❌ Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed...
|
|
🚀 Security Guard has started processing this pull request |
|
🔌 Smoke Services — All services reachable! ✅
|
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤
|
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 Warning Firewall blocked 7 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "accounts.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "msfeed25.pkgs.visualstudio.com"
- "update.googleapis.com"
- "www.google.com"
- "www.gstatic.com"See Network Configuration for more information.
|
|
✅ Build Test Suite completed successfully!
|
|
✅ Smoke Claude passed Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.anthropic.com"See Network Configuration for more information.
|
|
✅ Smoke Test: Copilot BYOK (Direct) Mode
Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy → api.githubcopilot.com Overall Status: ✅ PASS
|
|
EGRESS_RESULT allow=pass deny=pass ✅ Allowed domain (api.github.com) reachable — Overall status: PASS @lpcox network isolation egress enforcement working as expected. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "example.com"See Network Configuration for more information.
|
|
Smoke Test: Copilot Engine
Overall: PASS cc @lpcox
|
|
Smoke Test: Services Connectivity
Overall: PASS
|
Smoke Test: Claude Engine Validation
Overall result: PASS Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.anthropic.com"See Network Configuration for more information.
|
Chroot Version Comparison Results
Overall: FAILED — Node.js version mismatch between host and chroot environments. The
|
Smoke Test: API Proxy OpenTelemetry Tracing
Overall: 4/5 fully pass, 1 expected-pending (no OTLP endpoint configured in this environment) — no unexpected failures. One minor issue: the "Run api-proxy OTEL tests" step silently no-ops due to a deprecated
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — PASS Notes:
|
Smoke TestFix Cloud Hypervisor trusted artifact EACCES on hosted runners Warning Firewall blocked 7 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "accounts.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "msfeed25.pkgs.visualstudio.com"
- "update.googleapis.com"
- "www.google.com"
- "www.gstatic.com"See Network Configuration for more information.
|
Summary
Smoke Cloud Hypervisorto the standardready-for-awCI gateThe existing Cloud Hypervisor Copilot, Claude, Codex, build/test, and Playwright workflows provide direct counterparts to the removed Docker SBX workflows. This change keeps their specialized manual label triggers and makes the primary Cloud Hypervisor smoke replace the primary Docker SBX check in normal agentic CI.
Validation
gh aw compile smoke-cloud-hypervisornpx ts-node scripts/ci/postprocess-smoke-workflows.tsnpm test -- --runInBand scripts/ci/ready-for-aw-workflows.test.ts scripts/ci/postprocess-smoke-workflows.test.ts scripts/ci/test-cloud-hypervisor-workflow.test.ts scripts/ci/cloud-hypervisor-ci-scripts.test.tsnpm run buildRelated to #8834.