External threat detection fails on ARC/DinD: threat-detect is not staged into the chroot #59487

Missing binary staging: observed startup failure

• buildInstallThreatDetectStep, v0.88.6 passes only the version to the installer.
• install_threat_detect_binary.sh selects /usr/local/bin unless --rootless is supplied. Installation and the runner-side version check succeed in these runs.
• buildExternalDetectorPathSetup returns the Copilot PATH prefix on ARC/DinD without a host setup command. The existing Copilot copy step does not stage the separate detector binary.
• buildInstallAWFForExternalDetectorStep already derives installation mode from the detection sandbox configuration; the detector installer should follow the applicable policy too.

The runtime Compose artifact confirms a build-tools-populated sysroot:/host:rw volume, with the runner's shared ${RUNNER_TEMP}/gh-aw exposed both at its absolute path and at /host/tmp/awf-runner-bin. This is a separate chroot filesystem, not simply the runner's /usr/local/bin, nor literally the DinD daemon's root filesystem.

AWF's system mount builder exposes the configured binaries directory, and the entrypoint adds that overlay to PATH. Neither operation copies an unstaged runner-local threat-detect into it.

Adding --rootless alone is insufficient: changing installation to runner-local ~/.local/bin does not itself make the binary visible inside the chroot.

Path mismatch: code/configuration evidence, not a second runtime reproduction

The external command is built with /tmp/gh-aw/threat-detection in threat_detection_external.go, then ARC command rewriting changes those paths to ${RUNNER_TEMP}/gh-aw/threat-detection.

Meanwhile:

• Preparation defaults to /tmp/gh-aw/threat-detection.
• The explicit writable detection mount still uses /tmp/gh-aw/threat-detection.
• Artifact upload and conclusion also use the original /tmp/gh-aw/threat-detection/detection_result.json.

Relevant generated arguments, also present in the runtime Compose artifact:

--mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro"
--mount /tmp/gh-aw/threat-detection:/tmp/gh-aw/threat-detection:rw

threat-detect --engine copilot \
  --output ${RUNNER_TEMP}/gh-aw/threat-detection/detection_result.json \
  ${RUNNER_TEMP}/gh-aw/threat-detection

These are excerpts, not a complete shell command. The rewritten input directory has no corresponding preparation step, and the rewritten output lies under the read-only runtime mount. This must be covered when fixing startup, rather than claiming that copying the binary alone restores detection.

A concrete layout to validate is to prepare the full input directory under ${RUNNER_TEMP}/gh-aw/threat-detection, mount just that directory read-write, and collect the result back to the existing /tmp/gh-aw/threat-detection location before upload/conclusion. Missing or invalid results must retain the existing failure handling. Do not rewrite all agent paths globally or broaden the runtime mount to read-write.

Published versions inspected on 2026-09-08

Across these five tags, threat_detection_external.go, threat_detection_steps.go, threat_detection_job.go, awf_arc_dind.go, and the detector installer are individually byte-identical. For example, the external-detector file has Git blob SHA 478063ee7c6ac00605f9b9983ac5d32e1ddf65a9 throughout.

The bundled AWF upgrades were also inspected: the binary staging, system mount, volume builder, sysroot service and build-tools Dockerfile are unchanged across v0.28.12-v0.28.14. The changed browser-library staging, working-directory mounts and credential-overlay handling do not expose this runner-installed detector.

This is evidence that the relevant omissions remain in the released sources, not a claim of runtime reproduction on every prerelease.

Use a Linux ARC/DinD runner with a shared workspace/runtime directory visible to both runner and daemon. Replace arc-dind-runner in all four runner fields. The repository must support Copilot inference through copilot-requests: write.

Save the following as .github/workflows/repro-external-detection-arc.md. It deliberately provisions and stages Node to isolate this failure from #59235; this is reproduction scaffolding, not a proposed permanent runner workaround.

---
on:
  workflow_dispatch:
runs-on: arc-dind-runner
runs-on-slim: arc-dind-runner
runner:
  topology: arc-dind
permissions:
  contents: read
  copilot-requests: write
engine:
  id: copilot
  version: "1.0.80"
network: defaults
timeout-minutes: 10
jobs:
  detection:
    pre-steps:
      - name: Provision Node for detection reproduction
        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
        with:
          node-version: "24"
          package-manager-cache: false
      - name: Stage Node for detection reproduction
        shell: bash
        run: |
          set -euo pipefail
          NODE_BIN="$(command -v node)"
          NODE_PREFIX="$(dirname "$(dirname "$NODE_BIN")")"
          NODE_DEST="${RUNNER_TEMP}/gh-aw/tool-cache/detection-node"
          mkdir -p "$NODE_DEST"
          cp -a "$NODE_PREFIX"/. "$NODE_DEST"/
          test -x "$NODE_DEST/bin/node"
          echo "$NODE_DEST/bin" >> "$GITHUB_PATH"
          echo "GH_AW_NODE_BIN=$NODE_DEST/bin/node" >> "$GITHUB_ENV"
safe-outputs:
  runs-on: arc-dind-runner
  threat-detection:
    runs-on: arc-dind-runner
    continue-on-error: false
  create-issue:
    max: 1
---

Call the create_issue safe-output tool once with title
"External detector ARC reproduction" and body
"Benign output to exercise threat detection." Do not use noop.

Compile with gh-aw v0.88.2:

gh aw compile repro-external-detection-arc

Commit the source and generated lock file to a test repository and dispatch the workflow from that revision. Ensure the agent produces a non-empty safe output and RUN_DETECTION=true.

The expected pre-fix signature is a successful Node preflight followed by threat-detect: command not found, exit 127, and a missing result. The example intentionally uses continue-on-error: false so safe-output processing is blocked on failure; the original observations used the default true. Once fixed, this example can create one test issue.

This exact reduced example compiled successfully with gh-aw v0.88.2. It is separate from the two observed production runs and has not been dispatched as a new ARC runtime reproduction.

Use the existing test suites rather than adding a new runner:

• pkg/workflow/threat_detection_steps_test.go: installer invocation, install/stage/execute/collect/conclude ordering, and unchanged guard conditions.
• pkg/workflow/threat_detection_external_detector_execution_test.go: extend TestBuildExternalDetectorPathSetup and topology coverage to assert detector visibility, not just Copilot visibility.
• pkg/workflow/threat_detection_job_combinations_integration_test.go: external and inline paths, ARC and non-ARC, Copilot and other supported detector engines. Detector staging must not depend on whether the engine uses a standard or custom Copilot command.
• actions/setup/sh/install_threat_detect_binary_test.sh: retain checksum verification and cover rootless placement, executable permissions, failure propagation and runner-side availability.
• actions/setup/sh/prepare_threat_detection_files_test.sh: preserve all prepared inputs when using the shared destination, including prompt context, safe outputs and patches.
• pkg/workflow/threat_detection_conclude_script_test.go: skipped detection, successful results, missing results and both failure-policy modes.

In addition to compiler assertions, exercise the filesystem boundary on ARC/DinD: the runner's successful threat-detect --version is not proof that the sandbox can invoke it.

• The sandbox resolves and executes the staged detector.
• Prepared inputs are readable and the result destination is writable.
• A benign, non-empty input produces a valid structured result that upload and conclusion can find.
• Infrastructure failure produces no fabricated successful verdict and follows the configured policy.
• Raw detection.log remains excluded from the external detector artifact.
• Node availability remains intact, including when combined with the Node provisioning fix.
• Unrelated workflows retain their existing mounts, output contracts and failure defaults.

Run targeted regressions first, then make agent-finish in the project's Dev Container/Codespace. This issue proposes work for a core-team coding agent; it does not require an external-contributor PR.