Embed reviewed threat-detect digests in compiled workflows

[Copilot]

threat-detect binaries were verified against a checksum downloaded from the same release, leaving no independent trust root. Compiled workflows now carry reviewed release and digest pins directly.

• Compiler trust root
  • Pin threat-detect v0.5.2 and its complete release digest matrix.
  • Embed both Linux architecture digests in generated workflows.
  • Keep the release version and digest table as one review unit.
• Fail-closed installation
  • Verify downloaded binaries directly against compiler-supplied SHA-256 pins.
  • Reject missing, malformed, uppercase, or duplicate pins before downloading.
  • Remove runtime checksums.txt downloads.
  • Gate detector execution on successful verified installation, preventing fallback to preinstalled binaries.
• Artifact mirrors
  • Add an HTTPS-only artifact-base-url configuration.
  • Preserve compiler control of the release tag and expected digest when using a mirror.
  • Document the required <artifact-base-url>/<version>/threat-detect-linux-{amd64,arm64} layout, runner accessibility, unauthenticated access requirement, byte identity, and pin-update process.
• Maintenance
  • Document the detector pin update process.
  • Update generated workflow locks and release notes.

Fixes #61855
Fixes #57792

[Copilot]

Copilot review overview

🟡 Changes recommended

Installation failures can still invoke an unverified detector or silently omit the warn-mode failure conclusion.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 · 1

Open (2)

• Conclude path can run an unverified detector after install failure · New
• Test does not pin exact reviewed asset digests · New

What changed in this PR

Pins threat-detect releases and verifies downloaded binaries against compiler-controlled digests.

Changes:

• Adds v0.5.2 digest pins and HTTPS mirror configuration.
• Gates detector execution on installation success.
• Updates tests, documentation, generated locks, and release notes.

File	Description
pkg/​constants/​version_constants.go	Defines reviewed release pins.
pkg/​constants/​version_constants_test.go	Validates the pin matrix.
actions/​setup/​sh/​install_threat_detect_binary.sh	Verifies downloaded binaries.
actions/​setup/​sh/​install_threat_detect_binary_test.sh	Tests installer failure modes.
actions/​setup/​sh/​conclude_threat_detection.sh	Concludes detector execution.
pkg/​workflow/​threat_detection_external.go	Gates external detector execution.
pkg/​workflow/​threat_detection_steps.go	Emits installer arguments and pins.
pkg/​workflow/​threat_detection_config.go	Parses mirror configuration.
pkg/​workflow/​*threat_detection*_test.go	Covers compiler and conclusion behavior.
pkg/​parser/​schemas/​main_workflow_schema.json	Validates HTTPS mirror URLs.
pkg/​parser/​schema_threat_detection_test.go	Tests mirror schema validation.
docs/​src/​content/​docs/​reference/​threat-detection.md	Documents mirrors and pin maintenance.
.github/​workflows/​*.lock.yml	Embeds v0.5.2 Linux digests in generated workflows.
.changeset/​pin-threat-detect-release-digests.md	Records the patch release change.

[gh-aw-bot]

@copilot Please refresh this branch and continue the remaining implementation/review follow-up.

Run the pr-finisher skill after updating the branch and pushing the next changes.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 13 AIC · ⊞ 9.3K · ◷
_{Comment /souschef to run again}

[gh-aw-bot]

@copilot This PR still needs forward progress before a maintainer can merge it.

Please address the unresolved review feedback and then run the pr-finisher skill.

Unresolved review follow-ups:

• Copilot reviewer on pkg/constants/version_constants_test.go: Embed reviewed threat-detect digests in compiled workflows #61857 (comment)
• Copilot reviewer on pkg/workflow/threat_detection_external.go: Embed reviewed threat-detect digests in compiled workflows #61857 (comment)

I also requested a branch refresh for this PR.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "github.com"

See Network Configuration for more information.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 14.3 AIC · ⊞ 9.6K · ◷
_{Comment /souschef to run again}

[Copilot]

@copilot This PR still needs forward progress before a maintainer can merge it.

Please address the unresolved review feedback and then run the pr-finisher skill....

The cited findings are already addressed in 234ae94; the refreshed branch is ca03807. I completed the requested PR-finisher validation pass (make fmt, make lint, make test-unit, make test, and make recompile).

@copilot Please refresh this branch and continue the remaining implementation/review follow-up.

Run the pr-finisher skill after updating the branch and pushing the next changes....

Refreshed the branch with main in ca03807 and completed the requested PR-finisher validation pass.

[gh-aw-bot]

@copilot This PR still needs forward progress before a maintainer can merge it.

Please address the unresolved Copilot review feedback, refresh the branch if needed, and then run the pr-finisher skill before handing back to a maintainer.

Relevant review overview: #61857 (review)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "github.com"

See Network Configuration for more information.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 15.7 AIC · ⊞ 9.3K · ◷
_{Comment /souschef to run again}

[gh-aw-bot]

@copilot this PR has merge conflicts. Please run make merge-main to rebase/resolve conflicts, then continue with the pr-finisher skill so maintainers can re-review quickly.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "github.com"

See Network Configuration for more information.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 22.9 AIC · ⊞ 9.6K · ◷
_{Comment /souschef to run again}

[Copilot]

@copilot this PR has merge conflicts. Please run make merge-main to rebase/resolve conflicts, then continue with the pr-finisher skill so maintainers can re-review quickly.

[!WARNING]...

Resolved in 196810a. I ran make merge-main, regenerated the conflicted workflow locks, pushed the merge commit, and completed the PR-finisher local validation pass (fmt, lint, test-unit, test, recompile, plus agent-report-progress).

@copilot resolve the merge conflicts on this branch.

Resolved in 196810a by running make merge-main, resolving the generated workflow lock conflicts via recompilation, and pushing the two-parent merge commit against current main.

[gh-aw-bot]

pr-sous-chef
@copilot This PR still needs forward progress before a maintainer can merge it.

Please address the remaining review feedback below, refresh the branch if needed, and then run the pr-finisher skill.

Open review follow-ups (newest first):

• Copilot: Embed reviewed threat-detect digests in compiled workflows #61857 (comment)
• Copilot: Embed reviewed threat-detect digests in compiled workflows #61857 (comment)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "github.com"

See Network Configuration for more information.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 46.4 AIC · ⌖ 8.66 AIC · ⊞ 9.6K · ◷
_{Comment /souschef to run again}

[Copilot]

pr-sous-chef
@copilot This PR still needs forward progress before a maintainer can merge it.
...

Branch refreshed: merged current main (0eec778) in 48ffcfa, resolving the generated workflow lock conflicts by recompiling all 299 workflows.

The two listed review threads remain satisfied by 234ae94 and are unchanged in this head:

• pkg/workflow/threat_detection_external.go passes THREAT_DETECT_INSTALL_OUTCOME to the conclude step, and conclude_threat_detection.sh fails closed on any non-success install outcome before invoking threat-detect.
• pkg/constants/version_constants_test.go pins the exact reviewed v0.5.2 asset digests and asserts the complete release matrix.

Validation after the merge: make agent-report-progress (fmt, lint, impacted unit tests, recompile) passes with all lock files in sync.

[pelikhan]

@davidslater need a procedure/script/skill to upgrade to a newer release of threat detection

[gh-aw-bot]

pr-sous-chef
@copilot This PR still needs forward progress before a maintainer can merge it.

Please address the remaining review feedback below, refresh the branch if needed, and then run the pr-finisher skill.

Open review follow-ups (newest first):

• Copilot review overview: Embed reviewed threat-detect digests in compiled workflows #61857 (review)

Maintainer note to address as part of the next update:

• Embed reviewed threat-detect digests in compiled workflows #61857 (comment)

Run: https://github.com/github/gh-aw/actions/runs/35772888314

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 27.9 AIC · ⌖ 8.62 AIC · ⊞ 9.3K · ◷
_{Comment /souschef to run again}

[Copilot]

pr-sous-chef
@copilot This PR still needs forward progress before a maintainer can merge it.
...

Addressed in 1512631d: added make update-threat-detect-pins THREAT_DETECT_VERSION=vX.Y.Z as the maintainer-facing upgrade procedure, with version-shape validation before invoking the existing verified updater, then make fmt and make recompile. Updated /home/runner/work/gh-aw/gh-aw/docs/src/content/docs/reference/threat-detection.md to point maintainers at the make target. Verified: updater script tests, make target empty/malformed-version guards, make fmt && make lint && make test-unit && make test && make recompile, make agent-report-progress, secret scan, and final Code Review/CodeQL validation all pass. The referenced PR Sous Chef run had no failed jobs, and the two Copilot review threads are already resolved.

[github-actions]

🎉 This pull request is included in a new release.

Release: v0.89.20