Skip to content

feat(validation): grade runtime.host_containment from measured isolation - #1522

Open
burtenshaw wants to merge 5 commits into
ben/validation-netns-interfacesfrom
ben/rfc008-host-containment
Open

burtenshaw wants to merge 5 commits into
ben/validation-netns-interfacesfrom
ben/rfc008-host-containment

Conversation

@burtenshaw

@burtenshaw burtenshaw commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Implement runtime.host_containment from inspected Docker settings and a trusted helper running as the subject user in its PID namespace. Require a read-only image, inspect process confinement and the complete mount inventory, and reject subject-writable storage outside allotted roots.

launch + processes + mounts + bounded write checks -> measured containment
missing inventory or positive control -> SKIP

The device-filesystem exception is qualified with bounded subtree/file checks, including nested writable /dev storage. Request private cgroup namespaces and support both Docker init mount paths. Stacked on #1521.

Evidence

  • Before: a writable image root or nested writable device directory could evade containment.
  • After: real Docker fault/control regressions detect those cases while hardened public/no-network controls pass.

Test Plan

  • Final branch: 573 validation/CLI tests passed, 36 skipped.
  • Independent follow-up: 45 focused source tests and four targeted Docker cases passed.
  • Touched Python lint and diff checks passed.

Final top Linux Docker acceptance: 44/44 required cases passed at 986af419, with zero skips/errors/failures and complete artifact verification: https://github.com/openenv/OpenEnv/actions/runs/40746462903.

The main refresh adds only RFC 014 documentation; this member's runtime code and tests are unchanged.

RFC Status

RFC 008 / #778 remains In Review. The daemon-host read-sentinel positive control is deferred and explicitly unqualified; current evidence covers launch, process, mount and write containment.

Merge Danger

Door: two-way. Blast Radius: validation containment verdicts.

@burtenshaw burtenshaw added feature size: large Large pull request labels Oct 28, 2026 — with Cursor

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

Bugbot Autofix prepared fixes for both issues found in the latest run.

  • ✅ Fixed: Init mount path allowlist is incomplete
    • Added Docker Engine's /sbin/docker-init mount to the runtime allowlist and covered it in the containment fixture.
  • ✅ Fixed: Host cgroup namespace falsely fails check
    • The Docker provider now explicitly creates subjects with a private cgroup namespace across daemon defaults.

Create PR

Or push these changes by commenting:

@cursor push efc5d929a7
Preview (efc5d929a7)
diff --git a/src/openenv/validation/graders/runtime/containment.py b/src/openenv/validation/graders/runtime/containment.py
--- a/src/openenv/validation/graders/runtime/containment.py
+++ b/src/openenv/validation/graders/runtime/containment.py
@@ -12,6 +12,7 @@
     "/etc/hosts",
     "/etc/hostname",
     "/etc/resolv.conf",
+    "/sbin/docker-init",
     "/usr/sbin/docker-init",
     "/run/podman-init",
 }

diff --git a/src/openenv/validation/providers/docker.py b/src/openenv/validation/providers/docker.py
--- a/src/openenv/validation/providers/docker.py
+++ b/src/openenv/validation/providers/docker.py
@@ -347,6 +347,8 @@
             "--stop-timeout",
             "0",
             "--init",
+            "--cgroupns",
+            "private",
             "--user",
             "65532:65532",
             "--read-only",

diff --git a/tests/test_validation/test_docker_provider.py b/tests/test_validation/test_docker_provider.py
--- a/tests/test_validation/test_docker_provider.py
+++ b/tests/test_validation/test_docker_provider.py
@@ -119,6 +119,7 @@
         "--publish": "127.0.0.1::8000",
         "--network": "bridge",
         "--stop-timeout": "0",
+        "--cgroupns": "private",
     }.items():
         assert create[create.index(flag) + 1] == value
     assert "--read-only" in create and "--init" in create

diff --git a/tests/test_validation/test_host_containment.py b/tests/test_validation/test_host_containment.py
--- a/tests/test_validation/test_host_containment.py
+++ b/tests/test_validation/test_host_containment.py
@@ -53,6 +53,7 @@
             {"mount_point": "/dev/shm", "fstype": "tmpfs", "read_only": False},
             {"mount_point": "/sys/fs/cgroup", "fstype": "cgroup2", "read_only": True},
             {"mount_point": "/etc/hosts", "fstype": "ext4", "read_only": False},
+            {"mount_point": "/sbin/docker-init", "fstype": "ext4", "read_only": True},
             {"mount_point": "/tmp", "fstype": "tmpfs", "read_only": False},
         ],
         "allotted": ["/tmp", "/dev/shm"],

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit 1f858bd. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alignment Review Report

Automated Checks

  • Lint: PASS — usort check, ruff format --check, and ruff check are all clean on every file this PR touches (src/openenv/validation/graders/runtime/containment.py, src/openenv/validation/providers/_pidns.py, src/openenv/validation/providers/docker.py, src/openenv/validation/runtime/contracts.py, src/openenv/validation/runner.py, src/openenv/validation/types.py, and the new/updated tests). The repo-wide lint.sh run surfaces ~47 pre-existing unformatted files elsewhere in the tree (docs/README code blocks, cli/commands/*); none are touched by this diff.
  • Debug code: CLEAN — check-debug.sh flags one line in the new _pidns.py (print(json.dumps({...}))), but that's inside PROBE_SCRIPT, a string literal that becomes the body of a disposable python3 -c probe run in another container; the print is the probe's only output channel (consumed via json.loads(out) in docker.py), not leftover debug code. Everything else the hook finds is pre-existing and outside this diff.
  • Tests: tests/test_validation/test_host_containment.py (19 new tests) and the updated tests/test_validation/test_runner.py all pass (38/38). Ran the full non-Docker suite (pytest tests/ -m "not docker", 4026 passed / 5 failed / 161 skipped); the 5 failures (test_auto_env.py HF Hub installs, test_websockets.py echo-env step-count) are pre-existing and reproduce identically on the pre-PR commit (1e8e1f10) — unrelated to this change.

Open RFCs Context

  • RFC 008 — Environment Auto-Validation (In Review): this is the directly relevant RFC. It names runtime.host_containment as acceptance test #19 (level 2, local lane, fail severity) and its "Level 2 execution amendment" section specifies almost exactly this design: no privilege/host namespaces/host mounts, dropped capabilities, read-only image, bounded writable roots split between /tmp and /dev/shm, and "invasive containment and resource probes use disposable subjects." This PR implements that spec faithfully — the probe runs in its own --rm container, joins only the subject's PID namespace, drops all capabilities, and never steps the subject's own WebSocket session. The severity mapping for this check id already existed in severity-v1.json/severity-v2.json from an earlier PR and is untouched here, keeping the RFC's "graders have no opinion about severity, only the policy does" rule intact.
  • No other Draft/In-Review RFC (000, 001, 002, 003, 005, 006, 010, 011, 012) touches validation internals or this code path.

Tier 1: Fixes Required

None identified.

Tier 2: Alignment Discussion

Principle Conflicts

None identified. This PR only touches the src/openenv/validation/ developer-tooling subsystem; it doesn't touch the served Environment/Client API, the MCP/WebSocket boundary, or reward computation, so none of the INVARIANTS.md security/architectural invariants (agent isolation, dual API boundary, client-server separation, rewards-in-environment) are implicated.

RFC Conflicts

None identified — this change implements, rather than conflicts with, RFC 008's runtime.host_containment design (check #19), moves that check id from PENDING to IMPLEMENTED in the integration test inventory, and keeps its severity (fail, local lane) consistent with the RFC's acceptance-test table and the already-committed severity-v2.json.

Summary

  • 0 mechanical issues to fix
  • 0 alignment points for human review
  • 0 RFC conflicts to discuss

Nice implementation: the grader/evidence split (provider measures via ContainmentEvidence, grader judges without re-touching the subject) keeps with the registry's "graders read the manifest/evidence, never re-derive it" shape, and the test suite covers the FAIL-cannot-be-hidden-by-SKIP precedence explicitly (test_failed_control_cannot_hide_a_proven_violation).

Open in Web View Automation 

Sent by Cursor Automation: Pre-review

burtenshaw and others added 5 commits October 28, 2026 18:24
Inspect the launch from outside the subject and probe the server's processes
from the validator image, joined only to the subject's PID namespace and running
as the subject's user. The grader fails privileged launches, host namespaces,
added capabilities, devices, disabled confinement, root or capable processes,
unexpected mounts and writes outside /tmp and /dev/shm. An unproven write
control is a SKIP.

Part of RFC 008 Level Two (#778).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…table

runtime.host_containment passed a launch with a writable image root. The grader
ignored each mount's read-only flag, and the write probe tried only top-level
directories and working directories, so a world-writable /opt/nested/data under
root-owned parents went unnoticed. The probe now reads the whole mount table or
none of it, records each mount's device, read-only state and statvfs capacity, and
tries a write at every read-write mount outside the allotted roots. The grader
requires a read-only image root and fails any other writable filesystem except the
tmpfs allotted roots and the /proc, /dev/pts, /dev/mqueue and root-owned /dev
filesystems of a hardened Docker launch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Request a private cgroup namespace and recognize both Docker init mount paths. Exercise the full storage escape repro and describe measured containment and the deferred host-read control accurately.
A denied write at /dev does not rule out writable nested directories or regular files. Inventory the device filesystem from the trusted helper with bounded traversal and explicit incomplete evidence, and preserve positive storage witnesses.

Keep separate allotted and pseudo mounts outside the traversal. Add real Docker directory/file regressions and retain safe-launch controls.
An access denial on a subject-owned inode can be reversed with chmod. Mark those device inventories incomplete without changing existing inode permissions. Preserve stable root-owned denials and exercise both files and directories with real Docker controls.
@burtenshaw
burtenshaw force-pushed the ben/rfc008-host-containment branch from d404995 to 37a06a9 Compare October 28, 2026 17:25

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature size: large Large pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant