Skip to content

[aw-compat] Daily compatibility report: 9/17 repos fail strict-mode compile (2026-05-01) #29469

Closed as not planned
2 of 2 issues completed
Closed as not planned

Description

@github-actions

Overview

Daily cross-repo compatibility audit completed on 2026-05-01 using gh-aw ca92916.

  • Repos audited: 17 (all public repos confirmed to carry .github/workflows/*.lock.yml files)
  • Cloned OK: 17/17
  • Compile PASS after gh aw fix: 8/17 (47%)
  • Compile FAIL after gh aw fix: 9/17 (53%)

Note: GitHub code search (/search/code) was rate-limited throughout this run. All repositories were discovered via gh search repos, topic search (gh-aw), and manual enumeration of known gh-aw consumers. Full code-search-based discovery was not possible.

Failed Repositories

Repository Stars Error Cluster Auto-fixed?
github/gh-aw-mcpg 123 deprecated_sandbox_mcp_container, secret_expressions_in_agent_job No
github/gh-aw-firewall 62 deprecated_sandbox_mcp_version No
githubnext/gh-aw-test 7 deprecated_sandbox_agent_false (16 files) No
githubnext/gh-aw-trial-oxpecker-test 3 missing_persist_credentials_false No
githubnext/gh-aw-trial-FSharpStats-copilot-cli 2 missing_persist_credentials_false No
AgentMeter/agentmeter-action 2 ✅ passed —
rshade/gh-aw-fleet 2 ✅ passed —
githubnext/gh-aw-trial-oxpecker-perf 0 missing_persist_credentials_false No
githubnext/gh-aw-trial-FSarpStats-claude 0 missing_persist_credentials_false No
githubnext/gh-aw-trial-hono-copilot-cli 0 missing_persist_credentials_false No
az9713/gh-aw 0 deprecated_sandbox_mcp_container, invalid_engine_value, unknown_property_schema No

Error Cluster Analysis

Cluster 1: missing_persist_credentials_false — 5 repos

Pattern: actions/checkout steps in the agent job without persist-credentials: false

All 5 affected repos are githubnext trial repos created from the same template. The fix command migrated other deprecated fields (e.g. timeout_minutes → timeout-minutes, removed add-comment.discussion) but did not add persist-credentials: false.

Error message:

.github/workflows/daily-test-improver.md:1:1: error: strict mode: actions/checkout step(s) without
'persist-credentials: false' detected in the agent job: 'Checkout repository'. Without this setting
the git token is stored in .git/config and leaked to the agent.

Affected files (one per repo):

  • daily-test-improver.md
  • daily-perf-improver.md

Before:

steps:
  - name: Checkout repository
    uses: actions/checkout@v5

After (required):

steps:
  - name: Checkout repository
    uses: actions/checkout@v5
    with:
      persist-credentials: false
Cluster 2: deprecated_sandbox_mcp_container — 2 repos, 11 occurrences

Pattern: sandbox.mcp.container key is set to a custom container image

Repos: github/gh-aw-mcpg (11 files), az9713/gh-aw (fork, 3 files).

Error message:

error: strict mode: 'sandbox.mcp.container' is not allowed because it is an internal implementation
detail. Remove 'sandbox.mcp.container' or set 'strict: false' to disable strict mode.

Before:

sandbox:
  mcp:
    container: "ghcr.io/github/gh-aw-mcpg"

Workaround: Remove the key (container is auto-selected) or add strict: false.

Cluster 3: deprecated_sandbox_agent_false — 1 repo, 16 occurrences

Pattern: sandbox.agent: false disabling the sandbox firewall

Repo: githubnext/gh-aw-test (16 workflow files).

Error message:

error: strict mode: 'sandbox.agent: false' is not allowed because it disables the agent sandbox
firewall. This removes important security protections.

This test repo intentionally tests nosandbox scenarios. Suggested resolution: add strict: false per-workflow for legitimate testing workflows.

Cluster 4: deprecated_sandbox_mcp_version — 1 repo, 4 occurrences

Pattern: sandbox.mcp.version pinning an MCP server version

Repo: github/gh-aw-firewall (4 files).

Error message:

error: strict mode: 'sandbox.mcp.version' is not allowed because it is an internal implementation detail.

Before:

sandbox:
  mcp:
    version: v0.3.1

Workaround: Remove sandbox.mcp.version (version is auto-managed).

Additional errors in az9713/gh-aw (fork)
  • engine: custom is no longer a valid engine value. Valid engines: claude, codex, copilot, crush, gemini, opencode.
  • error_patterns is an unknown property at top level.
  • shared/*.md files have steps in wrong location (must be under on, secret-masking, or safe-outputs).

This fork was likely created before several schema changes and has accumulated multiple compatibility issues.

Key Observations

  1. gh aw fix does not fix the most common failure — The persist-credentials: false gap affects 5/9 failing repos and is not auto-resolved.
  2. Official GitHub repos also fail — github/gh-aw-mcpg and github/gh-aw-firewall both fail strict-mode compilation, suggesting these repos predate the sandbox.mcp.* strict-mode restriction.
  3. Trial repo template has a security gap — All trial repos share the same actions/checkout without security hardening, indicating the template source needs updating.

Recommendations

  • Add a gh aw fix codemod for persist-credentials: false (see separate issue).
  • Add a gh aw fix codemod to remove sandbox.mcp.container and sandbox.mcp.version when present.
  • Update the trial repo template to include persist-credentials: false in checkout steps.
  • Update github/gh-aw-mcpg, github/gh-aw-firewall, and githubnext/gh-aw-test to remove deprecated sandbox keys or add strict: false.

References:

Generated by Daily AW Cross-Repo Compile Check · ● 688.8K · ◷

  • expires on May 8, 2026, 8:30 AM UTC
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions