Skip to content

fix: hand off private Cloud Hypervisor diagnostics to runner ownership - #9659

Merged
lpcox merged 5 commits into
mainfrom
lpcox-cloud-hypervisor-audit-permissions
Oct 7, 2026
Merged

lpcox merged 5 commits into
mainfrom
lpcox-cloud-hypervisor-audit-permissions

Conversation

@lpcox

@lpcox lpcox commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

Symptom and evidence

The Smoke Cloud Hypervisor run 37688958250, validating #9559, failed in the shared gh-aw-actions secret-redaction step with ERR_VALIDATION: Failed to remove artifact source after secret redaction failed after replacing several non-writable audit files.

Cloud Hypervisor is invoked through sudo --preserve-env awf. Diagnostic producers created root-owned directories/files with modes 0700/0600. AWF's artifact cleanup then applied chmod -R a+rX to the audit tree, making these paths readable (0755/0644) without making them runner-writable; rootless ownership repair returns early when AWF is running as root. A readable root-owned diagnostic in a root-owned non-writable subdirectory can therefore be enumerated by the redactor but fail both replacement and deletion. The inaccessible-directory skip does not resolve that state.

The logs hide the exact rejected path and original redaction exception. This fixes an evidenced producer/handoff permissions defect consistent with the failure, not a confirmed identification of the failing file.

Changes

  • Resolve diagnostic ownership through the existing non-root host identity resolver, including sudo invocations. Chown the full diagnostic bundle, raw guest stdout/stderr audits, and preserved virtiofsd startup evidence to that identity.
  • Enforce directory 0700 and file 0600 modes, including existing files and boot-attempt/startup diagnostic roots.
  • Exclude Cloud Hypervisor diagnostic subtrees from legacy public-log chmod and rootless permission repair, across explicit audit paths and work-directory fallback preservation. Keep ordinary artifact permission behavior unchanged.
  • Document private, runner-owned diagnostic handoff.

Validation

  • npm run build — passed.
  • npm run type-check — passed.
  • Focused Jest run — 9 suites, 74 tests passed: diagnostic permissions, manager diagnostics/launch/virtiofsd/cleanup, artifact preservation permissions/errors/token usage, and rootless permission repair.
  • New regressions check requested chown UID/GID, restrictive on-disk modes, existing-file mode correction, startup/boot-attempt roots, ownership failure propagation, and collection-through-cleanup handoff across four artifact locations. Lifecycle tests perform real filesystem chmod/chown to the current test identity and verify subsequent file rewrite/removal; differing runner identity transfers are asserted through dependency mocks.
  • Changed-file ESLint passed with warnings only; git diff --check passed.

Live Linux/KVM smoke reproduction was not run on this macOS host. No workflows were manually dispatched.

Follow-up

A separate gh-aw-actions diagnostic improvement should report a safely sanitized path and failure stage/errno when redaction and cleanup both fail, without weakening fail-closed behavior. This PR does not change that external repository or generated gh-aw lock files.

Transfer diagnostic ownership to the invoking host identity and exclude private Cloud Hypervisor artifacts from legacy permission broadening during cleanup.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 7, 2026 22:31
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Documentation Preview

Documentation has been built for this PR.

Download preview artifact

To view locally:

  1. Download the docs-preview-pr-9659 artifact from the workflow run
  2. Unzip and open index.html in your browser

Built from commit 4297582

@lpcox

lpcox commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator Author

@copilot address review feedback

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Privileged diagnostic and cleanup operations can follow symlinks and modify unrelated host paths.

3 open findings
What changed in this PR

Hands Cloud Hypervisor diagnostics from privileged AWF processes to the non-root runner while preserving private permissions.

Changes:

  • Applies runner ownership and restrictive modes to diagnostic bundles.
  • Excludes private diagnostics from public artifact permission repair.
  • Adds regression coverage and documentation.
File Description
src/​cloud-hypervisor/​diagnostics.ts Adds diagnostic ownership handoff.
src/​cloud-hypervisor/​diagnostics-permissions.test.ts Tests ownership and modes.
src/​artifact-preservation.ts Preserves private diagnostic permissions.
src/​artifact-preservation-cloud-hypervisor.test.ts Tests cleanup handoff paths.
docs/​cloud-hypervisor-foundation.md Documents private runner-owned diagnostics.

🧠 Review effort: Balanced


Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

Comment thread src/artifact-preservation.ts Resolved
Comment thread src/cloud-hypervisor/diagnostics.ts Outdated
Comment on lines +219 to +221
await dependencies.mkdir(directory, { recursive: true, mode: 0o700 });
await dependencies.chmod(directory, 0o700);
await dependencies.chown(directory, identity.uid, identity.gid);
Comment thread src/cloud-hypervisor/diagnostics.ts Outdated
Comment on lines +231 to +233
await dependencies.writeFile(destination, contents, { mode: 0o600 });
await dependencies.chmod(destination, 0o600);
await dependencies.chown(destination, identity.uid, identity.gid);
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric Base PR Delta
Lines 93.01% 92.97% 📉 -0.04%
Statements 91.50% 91.47% 📉 -0.03%
Functions 90.21% 90.27% 📈 +0.06%
Branches 85.29% 85.22% 📉 -0.07%
📁 Per-file Coverage Changes (4 files)
File Lines (Before → After) Statements (Before → After)
src/cloud-hypervisor/diagnostics.ts 87.8% → 85.8% (-2.07%) 85.7% → 84.0% (-1.73%)
src/nvx/one-shot-adapter.ts 83.7% → 83.1% (-0.59%) 80.5% → 80.0% (-0.55%)
src/artifact-preservation.ts 94.8% → 95.0% (+0.15%) 95.0% → 94.7% (-0.26%)
src/log-directory-setup.ts 96.8% → 100.0% (+3.18%) 96.9% → 100.0% (+3.13%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot AI commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

@copilot address review feedback

Addressed the symlink-safety feedback in commit a6d79a1. Diagnostic directory components are checked before creation and revalidated before descriptor-based ownership/mode changes; diagnostic files use O_NOFOLLOW and same-descriptor operations; artifact permission repair skips symlink operands. Targeted tests pass. CodeQL reported no alerts but skipped analysis because the database is too large.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@lpcox Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@lpcox

lpcox commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator Author

Copilot AI commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

@copilot fix this failing codeql check https://github.com/github/gh-aw-firewall/pull/9659/checks?check_run_id=113056170980

Fixed the CodeQL findings in commit 9b0e3a4: the test now checks metadata with fstat on the same no-follow descriptor used for rewrite, and fallback artifacts are isolated under a unique private mkdtemp root. The focused test and type-check pass; the latest CodeQL analysis is pending for this commit.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric Base PR Delta
Lines 93.01% 92.97% 📉 -0.04%
Statements 91.50% 91.47% 📉 -0.03%
Functions 90.21% 90.27% 📈 +0.06%
Branches 85.29% 85.22% 📉 -0.07%
📁 Per-file Coverage Changes (5 files)
File Lines (Before → After) Statements (Before → After)
src/cloud-hypervisor/diagnostics.ts 87.8% → 85.8% (-2.07%) 85.7% → 84.0% (-1.73%)
src/nvx/one-shot-adapter.ts 83.7% → 83.1% (-0.59%) 80.5% → 80.0% (-0.55%)
src/enclave/host-executor-server.ts 90.3% → 90.3% (+0.00%) 88.8% → 89.1% (+0.31%)
src/artifact-preservation.ts 94.8% → 95.0% (+0.15%) 95.0% → 94.7% (-0.26%)
src/log-directory-setup.ts 96.8% → 100.0% (+3.18%) 96.9% → 100.0% (+3.13%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@lpcox

lpcox commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator Author

Copilot AI commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

@copilot the codeql check is still failing https://github.com/github/gh-aw-firewall/pull/9659/checks?check_run_id=113060378289

Updated the test in commit 048becd: test files are now created with exclusive O_NOFOLLOW and mode 0600, and the post-validation pathname unlink was removed. The focused tests/build/type-check pass. GitHub’s new CodeQL run is action_required and started no jobs, so its scan has not yet confirmed the alerts are cleared.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric Base PR Delta
Lines 93.01% 92.97% 📉 -0.04%
Statements 91.50% 91.47% 📉 -0.03%
Functions 90.21% 90.27% 📈 +0.06%
Branches 85.29% 85.22% 📉 -0.07%
📁 Per-file Coverage Changes (4 files)
File Lines (Before → After) Statements (Before → After)
src/cloud-hypervisor/diagnostics.ts 87.8% → 85.8% (-2.07%) 85.7% → 84.0% (-1.73%)
src/nvx/one-shot-adapter.ts 83.7% → 83.1% (-0.59%) 80.5% → 80.0% (-0.55%)
src/artifact-preservation.ts 94.8% → 95.0% (+0.15%) 95.0% → 94.7% (-0.26%)
src/log-directory-setup.ts 96.8% → 100.0% (+3.18%) 96.9% → 100.0% (+3.13%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric Base PR Delta
Lines 93.01% 92.97% 📉 -0.04%
Statements 91.50% 91.47% 📉 -0.03%
Functions 90.21% 90.27% 📈 +0.06%
Branches 85.29% 85.22% 📉 -0.07%
📁 Per-file Coverage Changes (4 files)
File Lines (Before → After) Statements (Before → After)
src/cloud-hypervisor/diagnostics.ts 87.8% → 85.8% (-2.07%) 85.7% → 84.0% (-1.73%)
src/nvx/one-shot-adapter.ts 83.7% → 83.1% (-0.59%) 80.5% → 80.0% (-0.55%)
src/artifact-preservation.ts 94.8% → 95.0% (+0.15%) 95.0% → 94.7% (-0.26%)
src/log-directory-setup.ts 96.8% → 100.0% (+3.18%) 96.9% → 100.0% (+3.13%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

🚀 Security Guard has started processing this pull request

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

💎 Smoke Gemini is crystallizing results on this pull request...

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

🔑 Smoke Copilot BYOK AOAI (api-key) is testing Azure OpenAI BYOK (api-key) mode on this pull request...

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

✅ Build Test Suite completed successfully!

Generated by Build Test Suite for #9659

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Smoke Test: Cloud Hypervisor + Copilot

  1. PASS — list_pull_requests call succeeded (returned PR data).
  2. PASS — https://github.com returned HTTP 200.
  3. PASS — Wrote and read back unique smoke file via printf/cat.
  4. PASS — (example.com/redacted) blocked (HTTP 000, curl exit 60).

All checks passed.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

EGRESS_RESULT allow=pass deny=pass

  • ✅ Allowed domain (api.github.com → 200)
  • ✅ Blocked domain (example.com blocked)

Overall: PASS — cc @lpcox

🛡️ Egress verdict from Smoke Copilot Network Isolation
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Smoke Test: Claude Engine Validation

  • API status: ✅ PASS
  • GH check: ✅ PASS
  • File status: ✅ PASS

Overall result: PASS

Generated by Smoke Claude for #9659 · claude · haiku45 · 45.4 AIC · ⊞ 6.1K · ◷
Add label ready-for-aw to run again

@github-actions github-actions Bot added smoke-copilot-network-isolation Copilot network-isolation egress smoke test smoke-claude labels Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Services smoke test

  • Redis PING: ✅ PONG
  • pg_isready: ✅ accepting connections
  • Postgres SELECT 1: ✅ 1

Overall: PASS

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot: PASS

  • ✅ GitHub MCP (last merged/closed PR: "fix: validate pinned NVX release on hosted KVM")
  • ✅ github.com connectivity (HTTP 200)
  • ✅ File write/read
    PR: "fix: hand off private Cloud Hypervisor diagnostics to runner ownership" — @lpcox

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct Mode) ✅

Status: PASS

Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY → api-proxy → api.githubcopilot.com)

  • ✅ GitHub MCP connectivity (2 recent merged PRs verified)
  • ✅ HTTP connectivity (code 200)
  • ✅ File write/read (smoke test file verified)
  • ✅ BYOK inference path (this response proves it works)

All tests passed at 2026-10-07T23:47:23Z

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Smoke OTEL: ✅ S1 otel.js loads (startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled...) · ✅ S2 68/68 OTEL tests pass (3 suites) · ✅ S3 env forwarding (GITHUB_AW_OTEL_TRACE_ID/PARENT_SPAN_ID in env-passthrough.ts; OTEL vars in api-proxy-env-config.ts) · ✅ S4 onUsage hook present in token-tracker-http.js · ✅ S5 otel.jsonl present (1 line recorded; file-export fallback).

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia ❌ not run ❌ FAIL
Bun hono ❌ not run ❌ FAIL
C++ fmt ✅ N/A ✅ PASS
C++ json ✅ N/A ✅ PASS
Deno oak ❌ not run ❌ FAIL
Deno std ❌ not run ❌ FAIL
.NET hello-world ✅ N/A ✅ PASS
.NET json-parse ✅ N/A ✅ PASS
Go color ✅ 1/1 passed ✅ PASS
Go env ✅ 1/1 passed ✅ PASS
Go uuid ✅ 1/1 passed ✅ PASS
Java gson ✅ 1/1 passed ✅ PASS
Java caffeine ✅ 1/1 passed ✅ PASS
Node.js clsx ✅ passed ✅ PASS
Node.js execa ✅ passed ✅ PASS
Node.js p-limit ✅ passed ✅ PASS
Rust fd ✅ 1/1 passed ✅ PASS
Rust zoxide ✅ 1/1 passed ✅ PASS

Overall: 6/8 ecosystems passed — FAIL

Failures

  • Bun / Deno: the installer commands (curl ... | bash / | sh) were denied by the sandbox permission layer ("Permission denied and could not request permission"). Bun and Deno are not on PATH, so no tests ran.
  • Java note: ~/.m2 was root-owned, so Maven could not create ~/.m2/repository by default. I used -Dmaven.repo.local=/tmp/gh-aw/agent/m2 as a workaround, and both projects then passed.

The build-test label was not added because of the failures.

Generated by Build Test Suite for #9659 · copilot · auto · 25 AIC · ⊞ 11.9K · ◷
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Chroot Version Comparison

Runtime Host Version Chroot Version Match?
Python Python 3.12.14 Python 3.12.14 ✅
Node.js v24.21.0 v22.23.2 ❌
Go go1.22.12 go1.22.12 ✅

Result: Not all tests passed. Node.js differs between host and chroot, so the smoke-chroot label was not added.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@lpcox
lpcox merged commit 0491c53 into main Oct 7, 2026
140 of 149 checks passed
@lpcox
lpcox deleted the lpcox-cloud-hypervisor-audit-permissions branch October 7, 2026 23:58

This branch had an error being deployed

1 failed deployment
aoai-model — e3c014d3 Deployed Oct 7, 2026 by lpcox via Conclusion #1907
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants