Skip to content

Follow-up to #7565: $GITHUB_WORKSPACE ownership is not repaired on native-root runners, so the agent cannot write to the checkout #7593

Closed

Description

@Adam-Smith-AER

Summary

#7565 fixed the ${RUNNER_TEMP}/gh-aw half of the native-root ownership problem,
and it works — thank you. On the same runner, the agent now starts, loads its MCP
servers, and runs a full session where previously it died after four EACCES
retries.

But the checkout is still root-owned while the agent runs as the fallback
sandbox identity (uid 1000), so the agent cannot write to $GITHUB_WORKSPACE. The
job then exits 0 and reports success while producing no output, which is
arguably worse than the original hard failure.

As with #7565, this is not a request to run the sandbox as root. The privilege
drop is correct. The ask is that the ownership repair introduced in #7565 also
cover the container workdir.

Environment

Runner AWS CodeBuild-hosted GitHub Actions runner (aws/codebuild/standard:7.0)
OS / user Ubuntu 22.04.5, uid 0 (root), no SUDO_UID, umask 0022
AWF v0.28.2 (contains #7565)
gh-aw v0.86.2, with sandbox.agent.version: v0.28.2 to pick up the fix
Engine claude
Docker 28.5.2, DOCKER_HOST unset, /var/run/docker.sock present

CodeBuild cannot be made non-root — AWS ignores the USER instruction in custom
images and documents no workaround ("Build commands run as root by default…
Recommended solution: None") — so "run the runner as non-root" is not available on
this platform.

Confirmation that #7565 works

The new diagnostic fires and the repair runs:

[WARN] Host process is running as root with no SUDO_UID; AWF will use sandbox
       identity 1000:1000. Host files not owned by that identity may be
       unreadable inside the sandbox unless AWF can repair ownership before launch.
[entrypoint] Transferred /host/tmp/gh-aw ownership to chroot user (1000:1000)

The MCP-config EACCES is gone. The agent ran 23 turns over 3m13s and exited 0.

The remaining failure

The agent reported this itself:

Blocker: every file and directory in the checkout (including .git) is
owned by root, but the agent runs as uid 1000. Creating
backend/pyproject.toml failed with EACCES, touch backend/.write-probe
failed with Permission denied, and git checkout -b agent/issue-61 failed
with fatal: detected dubious ownership.

Three distinct symptoms, one cause. The git "dubious ownership" error is a
symptom rather than the cause — safe.directory would silence the warning without
making the tree writable.

Root cause

The repair added in #7565 is scoped to the gh-aw root only —
repairRunnerTempGhAwOwnership() in src/config-writer.ts reads RUNNER_TEMP
and chowns ${RUNNER_TEMP}/gh-aw. The container workdir is not covered, even
though it is passed to AWF explicitly and mounted read-write:

awf --config … --container-workdir "${GITHUB_WORKSPACE}" …
MCP_GATEWAY_ALLOWED_MOUNT_ROOTS="${GITHUB_WORKSPACE}:rw,${RUNNER_TEMP}/gh-aw:ro,…"

So the workspace is writable by mount but not by ownership: root-owned tree,
uid-1000 writer.

Why this cannot be worked around downstream

For completeness, since the obvious workarounds were all tried:

  • No hook exists at the right point. gh-aw offers pre-steps (before the
    framework checkout) and post-steps (after the agent). The checkout happens
    after pre-steps, and there is no injection point between checkout and engine
    launch, so a workflow-level chown cannot be placed where it would help.
  • The agent cannot fix it itself. It runs as uid 1000 and cannot chown
    root-owned files.
  • The runner cannot be made non-root on this platform, as above.
  • Configuration cannot help. awf-config.schema.json enforces
    chroot.identity.uid: minimum 1, so an operator cannot align the sandbox
    identity with the host's root-owned files.

Secondary — and possibly the more important half

The run is a false green. With the workspace unwritable:

  • the agent step exits 0 (is_error: false, stop_reason: end_turn)
  • the agent job is success, and so is the overall run
  • no pull request is produced
  • the custom safe-job that would have released our issue-claim label is skipped,
    so the worked issue is left claimed

A scheduled or automated run in this state looks healthy while doing no work, and
costs a full agent session each time (~$0.91 in our case). Whatever happens with
the ownership fix, it would help enormously if an unwritable workspace failed
loudly — the same way #7565 added a warning for the identity fallback.

Suggested fix

  1. Extend the Repair gh-aw config ownership for native-root runners #7565 repair to the container workdir. Same function shape,
    same symlink-safe chown helper, applied to config.containerWorkDir when
    isNativeRootWithoutSudo() is true. This keeps the privilege drop, the
    read-only gh-aw mount, and the 0600 config mode untouched.
  2. Add a preflight writability check. After ownership repair, verify the
    sandbox identity can actually write to the workdir and fail with an actionable
    error if not. That converts a silent no-op run into a clear failure.

Happy to test a build on a CodeBuild runner — this reproduces 100% and we have the
infrastructure standing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions