feat: allow safe-outputs.threat-detection to be controlled by workflow_call inputs

docs/adr/29257-expression-controlled-threat-detection-for-workflow-call.md

@@ -0,0 +1,85 @@
+# ADR-29257: Expression-Controlled Threat Detection for `workflow_call` Reuse
+
+**Date**: 2026-04-30
+**Status**: Draft
+**Deciders**: pelikhan
+
+---
+
+## Part 1 — Narrative (Human-Friendly)
+
+### Context
+
+The `safe-outputs.threat-detection` field previously accepted only compile-time boolean or object values. This made it impossible for reusable `workflow_call` workflows to expose threat detection on/off control to callers via `inputs` — every combination required a separate workflow file. The same limitation applied to the `enabled` and `continue-on-error` sub-fields of the object form. The codebase had already established a pattern of accepting GitHub Actions expression strings for other parameterizable fields (PR #29212 for list constraints; PR #29230 for `protected-files` and `patch-format`); this PR applies the same pattern to the threat detection subsystem. Unlike those prior cases, threat detection is compiled as a separate GitHub Actions job rather than a handler configuration value, so allowing runtime control requires changes to the `if:` conditions on the `detection` job and on all downstream jobs that depend on its result (`safe_outputs`, `safe_jobs`, `upload_assets`).
+
+### Decision
+
+We will allow `safe-outputs.threat-detection` to accept a GitHub Actions expression string (matching `${{...}}`), and will allow the `enabled` and `continue-on-error` sub-fields of the object form to accept `templatable_boolean` (bool literal or expression string). When an expression is detected at parse time, the compiler stores it in `EnabledExpr` / `ContinueOnErrorExpr` fields of `ThreatDetectionConfig` and always emits the `detection` job (never skips compilation). The detection job's `if:` condition is extended with the raw caller expression so GitHub Actions evaluates it at runtime and skips the job when the expression resolves to `false`. Because a skipped `detection` job would cause downstream jobs that depend on it with a strict success condition to also be skipped silently, the `safe_outputs`, `safe_jobs`, and `upload_assets` jobs switch to `always() && (success || skipped)` semantics (`buildDetectionPassedCondition`) whenever the detection configuration is expression-controlled.
+
+### Alternatives Considered
+
+#### Alternative 1: Separate Workflow Files per Detection Mode
+
+Callers could maintain distinct workflow copies for each threat detection configuration. This was the status quo before this PR. It was rejected for the same reason as in ADR-29230: the duplication does not scale, and every structural change to the base workflow must be propagated to all variants, which teams routinely fail to do.
+
+#### Alternative 2: Add a New Dedicated `threat-detection-expression` Field
+
+A separate field (e.g., `threat-detection-enabled-expression`) could accept only expression strings while the existing field remained bool-only. This avoids changing the type of `threat-detection` but doubles the surface area and requires callers to know which field to use in which context. It was rejected as unnecessarily complex given that the `${{...}}`-pattern detection approach is already established in the codebase and yields a single, self-describing field.
+
+#### Alternative 3: Relax Gate Conditions on All Downstream Jobs Unconditionally
+
+Instead of conditionally switching downstream jobs to `always() && (success || skipped)` only when detection is expression-controlled, the gate condition could be relaxed for all workflows regardless of detection configuration. This was not chosen because it would weaken the strict-mode guarantee for workflows with static detection configurations, where a failed detection job must still block `safe_outputs`.
+
+### Consequences
+
+#### Positive
+- A single reusable `workflow_call` workflow can expose threat detection as a caller-controlled input without duplication.
+- Literal boolean and object forms remain fully backward-compatible; existing workflows are unaffected.
+- The `detection` job is always compiled when an expression is provided, ensuring the compiled lock file is static regardless of which runtime value the expression resolves to.
+- The fail-closed default (`always()` + `success || skipped` on downstream jobs) ensures that a misconfigured expression cannot silently skip safe-output handling.
+
+#### Negative
+- Expression values are only validated at runtime. A typo in a `workflow_call` input default (e.g., `default: ${{ inputs.typo }}`) will not be caught until the workflow executes.
+- All three downstream job builders (`buildConsolidatedSafeOutputsJob`, `buildSafeJobs`, `buildUploadAssetsJob`) must independently check `IsConditionalDetection()` and apply the extended condition, adding three separate call sites to keep in sync.
+- The `continue-on-error` step emitter (`buildDetectionConclusionStep`) must distinguish between a literal-true, literal-false, and expression value, adding a third branch to a function that previously had two.
+
+#### Neutral
+- The `extractRawExpression` helper was added to strip `${{` / `}}` wrappers from expression strings before embedding them in the YAML `if:` expression tree; this is a small pure utility with no broader side effects.
+- The `IsConditionalDetection` package-level helper provides a single authoritative check that is reused across all three affected compiler functions.
+- The `templatable_boolean` JSON schema `$ref` was already defined in the schema for other fields; this PR reuses it for `enabled` and `continue-on-error` without introducing a new definition.
+
+---
+
+## Part 2 — Normative Specification (RFC 2119)
+
+> The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**, **SHOULD**, **SHOULD NOT**, **RECOMMENDED**, **MAY**, and **OPTIONAL** in this section are to be interpreted as described in [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119).
+
+### Schema and Parsing
+
+1. The `safe-outputs.threat-detection` field **MUST** accept either a boolean literal, an object form, or a GitHub Actions expression string matching `^\$\{\{.*\}\}$`.
+2. The `enabled` and `continue-on-error` sub-fields of the object form **MUST** accept either a boolean literal or a GitHub Actions expression string (i.e., they **MUST** be typed as `templatable_boolean`).
+3. When `threat-detection` is a boolean literal `false`, or the object form sets `enabled: false`, the parser **MUST** return `nil` (detection disabled).
+4. When `threat-detection` is an expression string, the parser **MUST** store the full `${{...}}` string in `ThreatDetectionConfig.EnabledExpr` and **MUST NOT** treat the expression as a disabled state.
+5. When `continue-on-error` is an expression string, the parser **MUST** store it in `ThreatDetectionConfig.ContinueOnErrorExpr` and **MUST NOT** set `ContinueOnError` (the literal bool field) for that configuration.
+
+### Detection Job Compilation
+
+1. When `ThreatDetectionConfig.EnabledExpr` is non-nil, the compiler **MUST** always emit a `detection` job (it **MUST NOT** skip compiling the job based on the expression value).
+2. The `detection` job's `if:` condition **MUST** include the raw caller expression (the `${{...}}` wrappers stripped via `extractRawExpression`) appended with `&&` to the existing content guard condition.
+3. The `continue-on-error` step field on the detection conclusion step **MUST** emit the expression string verbatim (unquoted) when `ContinueOnErrorExpr` is non-nil, rather than a `"true"` or `"false"` literal.
+4. The `GH_AW_DETECTION_CONTINUE_ON_ERROR` environment variable on the detection conclusion step **MUST** emit the expression string verbatim when `ContinueOnErrorExpr` is non-nil.
+
+### Downstream Job Conditions
+
+1. When `IsConditionalDetection(data.SafeOutputs)` returns `true`, the `safe_outputs` job's `if:` condition **MUST** use `always() && <agent-not-skipped> && buildDetectionPassedCondition()` (accepting both `success` and `skipped` results from the detection job).
+2. When `IsConditionalDetection(data.SafeOutputs)` returns `true`, every `safe_jobs` custom job's base condition **MUST** be wrapped with `always() && <safe-output-type-check> && buildDetectionPassedCondition()`.
+3. When `IsConditionalDetection(data.SafeOutputs)` returns `true`, the `upload_assets` job's `if:` condition **MUST** use `always() && <upload-asset-check> && buildDetectionPassedCondition()`.
+4. When `IsConditionalDetection(data.SafeOutputs)` returns `false` and detection is statically enabled, downstream jobs **MUST** continue to use the strict `needs.detection.result == 'success'` gate (via `buildDetectionSuccessCondition()`).
+
+### Conformance
+
+An implementation is considered conformant with this ADR if it satisfies all **MUST** and **MUST NOT** requirements above. Failure to meet any **MUST** or **MUST NOT** requirement — in particular: skipping the detection job compilation for an expression-controlled config, failing to wrap downstream jobs with `always()` when detection is conditional, or emitting a quoted literal instead of the raw expression in step fields — constitutes non-conformance.
+
+---
+
+*This is a DRAFT ADR generated by the [Design Decision Gate](https://github.com/github/gh-aw/actions/runs/25170616496) workflow. The PR author must review, complete, and finalize this document before the PR can merge.*

pkg/cli/workflows/test-copilot-cache-memory-threat-detection-expression.md

@@ -0,0 +1,50 @@
+---
+description: Test workflow combining cache-memory with expression-controlled threat detection
+on:
+  workflow_call:
+    inputs:
+      enable-threat-detection:
+        description: 'Whether to enable threat detection at runtime'
+        type: boolean
+        default: true
+      task:
+        description: 'Task to store in cache'
+        type: string
+        default: 'Cache this result'
+
+permissions: read-all
+
+engine: copilot
+
+tools:
+  cache-memory:
+    retention-days: 7
+  github:
+    allowed: [get_repository]
+
+safe-outputs:
+  create-issue:
+    title-prefix: "[bot] "
+    labels: [automated]
+    max: 1
+  threat-detection: ${{ inputs.enable-threat-detection }}
+
+timeout-minutes: 10
+---
+
+# Test Cache Memory with Expression-Controlled Threat Detection
+
+This workflow demonstrates `cache-memory` combined with expression-controlled threat detection.
+The caller controls whether detection runs by passing `enable-threat-detection`.
+
+The compiled output must contain:
+- `detection` job with `if:` referencing `inputs.enable-threat-detection`
+- `actions/cache/restore` in the agent job (detection is present at compile time)
+- `update_cache_memory` job depending on `detection`
+- `update_cache_memory` condition using `always()` and accepting detection `skipped`
+  so cache is saved even when detection is skipped at runtime
+
+Steps:
+1. Check existing files in `/tmp/gh-aw/cache-memory/`
+2. Store a new entry: "Run ${{ github.run_number }}: ${{ inputs.task }}"
+3. Report a summary in a new issue

pkg/cli/workflows/test-copilot-cache-memory-threat-detection.md

@@ -0,0 +1,43 @@
+---
+description: Test workflow combining cache-memory with threat detection enabled
+on:
+  workflow_dispatch:
+    inputs:
+      task:
+        description: 'Task to store in cache'
+        required: true
+        default: 'Cache this result'
+
+permissions: read-all
+
+engine: copilot
+
+tools:
+  cache-memory:
+    retention-days: 7
+  github:
+    allowed: [get_repository]
+
+safe-outputs:
+  create-issue:
+    title-prefix: "[bot] "
+    labels: [automated]
+    max: 1
+  threat-detection: true
+
+timeout-minutes: 10
+---
+
+# Test Cache Memory with Threat Detection Enabled
+
+This workflow demonstrates `cache-memory` combined with standard threat detection.
+When detection is enabled the compiled output must contain:
+- `actions/cache/restore` (instead of `actions/cache`) in the agent job
+- An `update_cache_memory` job that depends on `detection`
+- `update_cache_memory` condition using `always()` and accepting detection `skipped`
+
+Steps:
+1. Check what files exist in `/tmp/gh-aw/cache-memory/` from previous runs
+2. Store a new entry: "Run ${{ github.run_number }}: ${{ inputs.task }}"
+3. Get basic repository information with the GitHub tool
+4. Report a summary in a new issue

pkg/cli/workflows/test-copilot-repo-memory-threat-detection-expression.md

@@ -0,0 +1,51 @@
+---
+description: Test workflow combining repo-memory with expression-controlled threat detection
+on:
+  workflow_call:
+    inputs:
+      enable-threat-detection:
+        description: 'Whether to enable threat detection at runtime'
+        type: boolean
+        default: true
+      task:
+        description: 'Task to store in memory'
+        type: string
+        default: 'Record this run'
+
+permissions: read-all
+
+engine: copilot
+
+tools:
+  repo-memory:
+    branch-name: memory/test-threat-detection-expr
+    description: "Test repo-memory with expression-controlled threat detection"
+    max-file-size: 524288
+  github:
+    allowed: [get_repository]
+
+safe-outputs:
+  create-issue:
+    title-prefix: "[bot] "
+    labels: [automated]
+    max: 1
+  threat-detection: ${{ inputs.enable-threat-detection }}
+
+timeout-minutes: 10
+---
+
+# Test Repo Memory with Expression-Controlled Threat Detection
+
+This workflow demonstrates `repo-memory` combined with expression-controlled threat detection.
+The caller controls whether detection runs by passing `enable-threat-detection`.
+
+The compiled output must contain:
+- `detection` job with `if:` referencing `inputs.enable-threat-detection`
+- `push_repo_memory` job depending on `detection`
+- `push_repo_memory` condition using `always()` and accepting detection `skipped`
+  so that the memory is persisted even when detection is skipped at runtime
+
+Steps:
+1. Check existing files in `/tmp/gh-aw/repo-memory-default/memory/default/`
+2. Append a new entry: "Run ${{ github.run_number }}: ${{ inputs.task }}"
+3. Report a summary in a new issue

pkg/cli/workflows/test-copilot-repo-memory-threat-detection.md

@@ -0,0 +1,44 @@
+---
+description: Test workflow combining repo-memory with threat detection enabled
+on:
+  workflow_dispatch:
+    inputs:
+      task:
+        description: 'Task to store in memory'
+        required: true
+        default: 'Record this run'
+
+permissions: read-all
+
+engine: copilot
+
+tools:
+  repo-memory:
+    branch-name: memory/test-threat-detection
+    description: "Test repo-memory with threat detection"
+    max-file-size: 524288
+  github:
+    allowed: [get_repository]
+
+safe-outputs:
+  create-issue:
+    title-prefix: "[bot] "
+    labels: [automated]
+    max: 1
+  threat-detection: true
+
+timeout-minutes: 10
+---
+
+# Test Repo Memory with Threat Detection Enabled
+
+This workflow demonstrates `repo-memory` combined with threat detection enabled.
+The compiled output must contain:
+- `push_repo_memory` job depending on `detection`
+- `push_repo_memory` job condition using `always()` and accepting detection `skipped`
+
+Steps:
+1. Check what files exist in `/tmp/gh-aw/repo-memory-default/memory/default/` from prior runs
+2. Append a new entry: "Run ${{ github.run_number }}: ${{ inputs.task }}"
+3. Get basic repository information with the GitHub tool
+4. Report a summary in a new issue

pkg/cli/workflows/test-copilot-safe-jobs-threat-detection-expression.md

@@ -0,0 +1,48 @@
+---
+description: Test workflow combining safe-jobs with expression-controlled threat detection
+on:
+  workflow_call:
+    inputs:
+      enable-threat-detection:
+        description: 'Whether to enable threat detection at runtime'
+        type: boolean
+        default: true
+
+permissions:
+  contents: read
+  actions: read
+
+engine: copilot
+
+safe-outputs:
+  threat-detection: ${{ inputs.enable-threat-detection }}
+  jobs:
+    summarize:
+      runs-on: ubuntu-latest
+      steps:
+        - name: Print summary
+          run: |
+            if [ -f "$GH_AW_AGENT_OUTPUT" ]; then
+              echo "Agent output available"
+              cat "$GH_AW_AGENT_OUTPUT"
+            else
+              echo "No agent output found"
+            fi
+
+timeout-minutes: 10
+---
+
+# Test Safe Jobs with Expression-Controlled Threat Detection
+
+This workflow demonstrates `safe-outputs.jobs` combined with expression-controlled threat detection.
+
+When `enable-threat-detection` is `true` (default) the detection job runs before `safe_jobs`.
+When `enable-threat-detection` is `false` the detection job is skipped and `safe_jobs`
+still runs because its condition accepts detection `skipped`.
+
+The compiled output must contain:
+- `detection` job with `if:` referencing `inputs.enable-threat-detection`
+- `safe_jobs` (the custom `summarize` job) depending on `detection`
+- `safe_jobs` condition using `always()` and accepting detection `skipped`
+
+Produce a short text output (one sentence describing the repository) using the summarize tool.

pkg/cli/workflows/test-copilot-threat-detection-expression.md

@@ -0,0 +1,43 @@
+---
+description: Test workflow for expression-controlled threat detection via workflow_call inputs
+on:
+  workflow_call:
+    inputs:
+      enable-threat-detection:
+        description: 'Whether to enable threat detection'
+        type: boolean
+        default: true
+      detection-continue-on-error:
+        description: 'Whether threat detection failures are non-blocking'
+        type: boolean
+        default: false
+
+permissions: read-all
+
+engine: copilot
+
+safe-outputs:
+  create-issue:
+    title-prefix: "[bot] "
+    labels: [automated]
+    max: 1
+  # Object form: exercises both 'enabled' and 'continue-on-error' as expressions.
+  # The shorthand form (threat-detection: ${{ expr }}) is covered by the
+  # *-threat-detection-expression.md fixtures and the integration tests.
+  threat-detection:
+    enabled: ${{ inputs.enable-threat-detection }}
+    continue-on-error: ${{ inputs.detection-continue-on-error }}
+
+timeout-minutes: 10
+---
+
+# Test Expression-Controlled Threat Detection
+
+This workflow demonstrates parameterising `safe-outputs.threat-detection` at runtime
+via `workflow_call` inputs so callers can toggle detection without separate workflow files.
+
+When called with `enable-threat-detection: true` the detection job runs normally.
+When called with `enable-threat-detection: false` the detection job is skipped and
+`safe_outputs` still runs (its condition accepts `detection.result == 'skipped'`).
+
+Create a brief summary issue noting whether threat detection was enabled or skipped.