Threat-detection job silently no-ops on arc-dind runners: Copilot binary not staged into the chroot (spawn /usr/local/bin/copilot ENOENT) #44249

Summary

With runner.topology: arc-dind (ARC + Docker-in-Docker sidecar), the agent job works correctly (after #44145), but the threat-detection job silently no-ops: its Copilot CLI execution fails with spawn /usr/local/bin/copilot ENOENT inside the AWF chroot, the failure is swallowed by the detection step's continue-on-error: true, and the job reports success while the detection model never runs.

Net effect: threat detection provides no protection on arc-dind runners while appearing green. This is a security-relevant silent failure.

Root cause: the detection job constructs a fresh minimal WorkflowData that does not carry RunnerConfig, so every isArcDindTopology()-gated daemon-visibility behavior (Copilot staging step + daemon-visible spawn path) is skipped for the detection job only.

Environment

• gh-aw v0.82.4 (AWF v0.27.27, mcpg v0.4.0)
• Engine: GitHub Copilot CLI (engine: copilot)
• Runner: actions-runner-controller gha-runner-scale-set with a privileged dind sidecar (split filesystem), runner: topology: arc-dind
• safe-outputs.threat-detection enabled, inline detection engine (default; gh-aw-detection feature flag off), squid network-isolation

Symptom

The detection job is green, but its log shows the engine never started:

[copilot-harness] pre-flight: command not found: /usr/local/bin/copilot (F_OK check failed — binary does not exist at this path)
[copilot-harness] attempt 1: failed to start process '/usr/local/bin/copilot': spawn /usr/local/bin/copilot ENOENT
[WARN] Command completed with exit code: 1
##[error]Process completed with exit code 1.
📋 continue-on-error: true
##[error]❌ Failed to parse detection result: No THREAT_DETECTION_RESULT found in detection log.

The execution step is marked continue-on-error: true (intended to tolerate transient infra/API failures), so the ENOENT never fails the job and detection silently yields no result.

Root cause (source analysis — pkg/workflow)

The Copilot binary is handled in two places, both gated on isArcDindTopology(workflowData):

1. Daemon-visible staging step — nodejs.go, BuildNpmEngineInstallStepsWithAWF:

   if isFirewallEnabled(workflowData) && isArcDindTopology(workflowData) {
       // emits "Copy Copilot CLI to daemon-visible path":
       //   cp /usr/local/bin/copilot "${RUNNER_TEMP}/gh-aw/bin/copilot"
   }

2. Spawn path — copilot_engine_execution.go, resolveCopilotCommand:

   if isArcDindTopology(workflowData) {
       return constants.GhAwRootDirShell + "/bin/copilot", ""   // daemon-visible copy
   }
   return constants.CopilotBinaryPath, ""                        // /usr/local/bin/copilot

isArcDindTopology → getRunnerTopology(workflowData) → workflowData.RunnerConfig.Topology, which returns "" when RunnerConfig == nil (awf_config.go).

The detection job builds a new, minimal WorkflowData in buildDetectionEngineExecutionStep (threat_detection_inline_engine.go) and omits RunnerConfig:

threatDetectionData := &WorkflowData{
    Tools:             map[string]any{"bash": []any{"*"}},
    SafeOutputs:       nil,
    EngineConfig:      detectionEngineConfig,
    AI:                engineSetting,
    Features:          data.Features,
    Permissions:       data.Permissions,
    CachedPermissions: data.CachedPermissions,
    IsDetectionRun:    true,
    NetworkPermissions: &NetworkPermissions{ Allowed: getThreatDetectionAdditionalAllowedDomains(data) },
    SandboxConfig:     &SandboxConfig{ Agent: &AgentSandboxConfig{ Type: SandboxTypeAWF } },
    // RunnerConfig is NOT copied from `data`  ← the bug
}
...
installSteps   := engine.GetInstallationSteps(threatDetectionData)      // → no staging step
executionSteps := engine.GetExecutionSteps(threatDetectionData, logFile) // → spawns /usr/local/bin/copilot

So for the detection job only, isArcDindTopology(threatDetectionData) == false: no "Copy Copilot CLI to daemon-visible path" step is emitted, and the engine is spawned from /usr/local/bin/copilot. But the detection engine still runs inside the AWF chroot (SandboxConfig.Agent.Type = AWF), whose filesystem is the dind daemon's — where /usr/local/bin/copilot does not exist. → spawn ENOENT.

(Some arc-dind handling is applied to detection — e.g. the tcp://-gated chroot binaries/identity patch is present in the compiled detection job — which is what makes the missing RunnerConfig in the Copilot-path resolution the specific gap.)

Evidence (same frontmatter, compiled lock)

Reproduction

1. Compile any engine: copilot workflow with runner: topology: arc-dind and safe-outputs.threat-detection enabled (inline engine; gh-aw-detection feature off).
2. Run it on an ARC runner with a dind sidecar, where the AWF chroot executes on the dind daemon's filesystem (Copilot installed to /usr/local/bin on the runner is not present there).
3. Agent job succeeds; detection job is green, but its log shows spawn /usr/local/bin/copilot ENOENT and No THREAT_DETECTION_RESULT found.

Proposed fix + implementation plan

Propagate the runner topology into the detection WorkflowData so the detection engine's install and execution use the same arc-dind daemon-visibility handling as the agent job.

1. pkg/workflow/threat_detection_inline_engine.go — in buildDetectionEngineExecutionStep, carry the topology into the constructed threatDetectionData:

   threatDetectionData := &WorkflowData{
       ...
       RunnerConfig: data.RunnerConfig, // propagate runner.topology (e.g. arc-dind) to the detection job
   }

   This flips isArcDindTopology(threatDetectionData) to true on arc-dind, so engine.GetInstallationSteps emits the "Copy Copilot CLI to daemon-visible path" step and resolveCopilotCommand returns the daemon-visible ${RUNNER_TEMP}/gh-aw/bin/copilot.

2. Resolve the rootless/sudo interaction. The detection execution currently runs sudo -E awf … --enable-host-access, whereas arc-dind agent execution is rootless (sudo: false; enforced by validateArcDindRootless in runner_topology_validation.go). Decide whether the detection job under arc-dind should also run rootless (consistent with the agent) and adjust the detection AWF command so validateArcDindRootless passes for the detection steps. This design decision is surfaced by step 1 and should be handled in the same change.

3. Chroot mounts. Confirm the detection chroot receives the same daemon-visible mounts as the agent (workspace, ${RUNNER_TEMP}/gh-aw, tool-cache) so the staged Copilot and its Node runtime are reachable.

4. Tests (pkg/workflow/threat_detection_test.go; mirror the agent assertions in compiler_yaml_main_job_test.go):

   • Given runner: topology: arc-dind + engine: copilot + threat-detection enabled, assert the compiled detection job (a) contains a "Copy Copilot CLI to daemon-visible path" step and (b) passes ${RUNNER_TEMP}/gh-aw/bin/copilot to copilot_harness.cjs (not /usr/local/bin/copilot).
   • Regression: without arc-dind, the detection job still uses /usr/local/bin/copilot.

5. Make the silent failure visible (defense-in-depth): when the detection engine step exits non-zero with no output under continue-on-error, emit a warning/annotation from the parse-and-conclude step (the No THREAT_DETECTION_RESULT found branch) so a broken detection engine is surfaced rather than reported as a green success.

6. Docs: note arc-dind support for the threat-detection job in docs/src/content/docs/reference/threat-detection.md.

7. Run make agent-finish (build, test, recompile, format, lint) before completing.