Repository navigation
feat(validation): grade runtime.host_containment from measured isolation - #1522
burtenshaw wants to merge 5 commits into
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Init mount path allowlist is incomplete
- Added Docker Engine's /sbin/docker-init mount to the runtime allowlist and covered it in the containment fixture.
- ✅ Fixed: Host cgroup namespace falsely fails check
- The Docker provider now explicitly creates subjects with a private cgroup namespace across daemon defaults.
Or push these changes by commenting:
@cursor push efc5d929a7
Preview (efc5d929a7)
diff --git a/src/openenv/validation/graders/runtime/containment.py b/src/openenv/validation/graders/runtime/containment.py
--- a/src/openenv/validation/graders/runtime/containment.py
+++ b/src/openenv/validation/graders/runtime/containment.py
@@ -12,6 +12,7 @@
"/etc/hosts",
"/etc/hostname",
"/etc/resolv.conf",
+ "/sbin/docker-init",
"/usr/sbin/docker-init",
"/run/podman-init",
}
diff --git a/src/openenv/validation/providers/docker.py b/src/openenv/validation/providers/docker.py
--- a/src/openenv/validation/providers/docker.py
+++ b/src/openenv/validation/providers/docker.py
@@ -347,6 +347,8 @@
"--stop-timeout",
"0",
"--init",
+ "--cgroupns",
+ "private",
"--user",
"65532:65532",
"--read-only",
diff --git a/tests/test_validation/test_docker_provider.py b/tests/test_validation/test_docker_provider.py
--- a/tests/test_validation/test_docker_provider.py
+++ b/tests/test_validation/test_docker_provider.py
@@ -119,6 +119,7 @@
"--publish": "127.0.0.1::8000",
"--network": "bridge",
"--stop-timeout": "0",
+ "--cgroupns": "private",
}.items():
assert create[create.index(flag) + 1] == value
assert "--read-only" in create and "--init" in create
diff --git a/tests/test_validation/test_host_containment.py b/tests/test_validation/test_host_containment.py
--- a/tests/test_validation/test_host_containment.py
+++ b/tests/test_validation/test_host_containment.py
@@ -53,6 +53,7 @@
{"mount_point": "/dev/shm", "fstype": "tmpfs", "read_only": False},
{"mount_point": "/sys/fs/cgroup", "fstype": "cgroup2", "read_only": True},
{"mount_point": "/etc/hosts", "fstype": "ext4", "read_only": False},
+ {"mount_point": "/sbin/docker-init", "fstype": "ext4", "read_only": True},
{"mount_point": "/tmp", "fstype": "tmpfs", "read_only": False},
],
"allotted": ["/tmp", "/dev/shm"],You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit 1f858bd. Configure here.
There was a problem hiding this comment.
Alignment Review Report
Automated Checks
- Lint: PASS —
usort check,ruff format --check, andruff checkare all clean on every file this PR touches (src/openenv/validation/graders/runtime/containment.py,src/openenv/validation/providers/_pidns.py,src/openenv/validation/providers/docker.py,src/openenv/validation/runtime/contracts.py,src/openenv/validation/runner.py,src/openenv/validation/types.py, and the new/updated tests). The repo-widelint.shrun surfaces ~47 pre-existing unformatted files elsewhere in the tree (docs/README code blocks,cli/commands/*); none are touched by this diff. - Debug code: CLEAN —
check-debug.shflags one line in the new_pidns.py(print(json.dumps({...}))), but that's insidePROBE_SCRIPT, a string literal that becomes the body of a disposablepython3 -cprobe run in another container; theprintis the probe's only output channel (consumed viajson.loads(out)indocker.py), not leftover debug code. Everything else the hook finds is pre-existing and outside this diff. - Tests:
tests/test_validation/test_host_containment.py(19 new tests) and the updatedtests/test_validation/test_runner.pyall pass (38/38). Ran the full non-Docker suite (pytest tests/ -m "not docker", 4026 passed / 5 failed / 161 skipped); the 5 failures (test_auto_env.pyHF Hub installs,test_websockets.pyecho-env step-count) are pre-existing and reproduce identically on the pre-PR commit (1e8e1f10) — unrelated to this change.
Open RFCs Context
- RFC 008 — Environment Auto-Validation (In Review): this is the directly relevant RFC. It names
runtime.host_containmentas acceptance test #19 (level 2, local lane, fail severity) and its "Level 2 execution amendment" section specifies almost exactly this design: no privilege/host namespaces/host mounts, dropped capabilities, read-only image, bounded writable roots split between/tmpand/dev/shm, and "invasive containment and resource probes use disposable subjects." This PR implements that spec faithfully — the probe runs in its own--rmcontainer, joins only the subject's PID namespace, drops all capabilities, and never steps the subject's own WebSocket session. The severity mapping for this check id already existed inseverity-v1.json/severity-v2.jsonfrom an earlier PR and is untouched here, keeping the RFC's "graders have no opinion about severity, only the policy does" rule intact. - No other Draft/In-Review RFC (000, 001, 002, 003, 005, 006, 010, 011, 012) touches validation internals or this code path.
Tier 1: Fixes Required
None identified.
Tier 2: Alignment Discussion
Principle Conflicts
None identified. This PR only touches the src/openenv/validation/ developer-tooling subsystem; it doesn't touch the served Environment/Client API, the MCP/WebSocket boundary, or reward computation, so none of the INVARIANTS.md security/architectural invariants (agent isolation, dual API boundary, client-server separation, rewards-in-environment) are implicated.
RFC Conflicts
None identified — this change implements, rather than conflicts with, RFC 008's runtime.host_containment design (check #19), moves that check id from PENDING to IMPLEMENTED in the integration test inventory, and keeps its severity (fail, local lane) consistent with the RFC's acceptance-test table and the already-committed severity-v2.json.
Summary
- 0 mechanical issues to fix
- 0 alignment points for human review
- 0 RFC conflicts to discuss
Nice implementation: the grader/evidence split (provider measures via ContainmentEvidence, grader judges without re-touching the subject) keeps with the registry's "graders read the manifest/evidence, never re-derive it" shape, and the test suite covers the FAIL-cannot-be-hidden-by-SKIP precedence explicitly (test_failed_control_cannot_hide_a_proven_violation).
Sent by Cursor Automation: Pre-review
1f858bd to
d404995
Compare
1e8e1f1 to
14693bc
Compare
Inspect the launch from outside the subject and probe the server's processes from the validator image, joined only to the subject's PID namespace and running as the subject's user. The grader fails privileged launches, host namespaces, added capabilities, devices, disabled confinement, root or capable processes, unexpected mounts and writes outside /tmp and /dev/shm. An unproven write control is a SKIP. Part of RFC 008 Level Two (#778). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…table runtime.host_containment passed a launch with a writable image root. The grader ignored each mount's read-only flag, and the write probe tried only top-level directories and working directories, so a world-writable /opt/nested/data under root-owned parents went unnoticed. The probe now reads the whole mount table or none of it, records each mount's device, read-only state and statvfs capacity, and tries a write at every read-write mount outside the allotted roots. The grader requires a read-only image root and fails any other writable filesystem except the tmpfs allotted roots and the /proc, /dev/pts, /dev/mqueue and root-owned /dev filesystems of a hardened Docker launch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Request a private cgroup namespace and recognize both Docker init mount paths. Exercise the full storage escape repro and describe measured containment and the deferred host-read control accurately.
A denied write at /dev does not rule out writable nested directories or regular files. Inventory the device filesystem from the trusted helper with bounded traversal and explicit incomplete evidence, and preserve positive storage witnesses. Keep separate allotted and pseudo mounts outside the traversal. Add real Docker directory/file regressions and retain safe-launch controls.
An access denial on a subject-owned inode can be reversed with chmod. Mark those device inventories incomplete without changing existing inode permissions. Preserve stable root-owned denials and exercise both files and directories with real Docker controls.
d404995 to
37a06a9
Compare




Summary
Implement
runtime.host_containmentfrom inspected Docker settings and a trusted helper running as the subject user in its PID namespace. Require a read-only image, inspect process confinement and the complete mount inventory, and reject subject-writable storage outside allotted roots.The device-filesystem exception is qualified with bounded subtree/file checks, including nested writable
/devstorage. Request private cgroup namespaces and support both Docker init mount paths. Stacked on #1521.Evidence
Test Plan
Final top Linux Docker acceptance: 44/44 required cases passed at
986af419, with zero skips/errors/failures and complete artifact verification: https://github.com/openenv/OpenEnv/actions/runs/40746462903.The main refresh adds only RFC 014 documentation; this member's runtime code and tests are unchanged.
37a06a90, including locked suites, exact-wheel Docker acceptance and artifact verification: https://github.com/openenv/OpenEnv/actions/runs/40735608047.RFC Status
RFC 008 / #778 remains In Review. The daemon-host read-sentinel positive control is deferred and explicitly unqualified; current evidence covers launch, process, mount and write containment.
Merge Danger
Door: two-way. Blast Radius: validation containment verdicts.