Repository navigation
[deep-report] PR Sous Chef: daily AIC guardrail cascading failure from one run's missing usage-accounting file #61137
Description
Problem
The PR Sous Chef workflow has failed 100% of its scheduled runs since ~2026-09-15T10:43Z (7/7 in a spot-checked window). Root cause, confirmed by reading the activation job logs for run 34969377719:
[daily-workflow-aic] AIC inspection failed: {"status":"transient_error","error":"Missing accounting for executed agent component in run 34952385840 (attempt 1, job 104326484539, conclusion failure): agent/token_usage.jsonl is missing; agent_usage.jsonl is missing; agent_usage.json is missing"}
##[error]Daily workflow AI Credits are unknown: ...
A single earlier run (34952385840, itself already reported as a workflow failure in issue #61081/#61082) had its agent job fail before writing any usage-accounting file. The daily-AIC guardrail (actions/setup/js/check_daily_aic_workflow_guardrail.cjs, invoked by activation) walks back over the last 24h of runs to total AI Credit usage, and sumCoveredComponents in actions/setup/js/daily_aic_component_coverage.cjs throws when it can't prove coverage for a completed, non-skipped component job. That "unrecoverable" throw fails activation for every subsequent run that includes run 34952385840 in its 24h lookback window — not just the poisoned run itself.
The code already has a carve-out for exactly this scenario, but it's incomplete: daily_aic_component_coverage.cjs:152 treats a failed agent job as zero-cost ("failed_before_accounting") only when the primary accounting file exists but is empty (candidateStates[0].state === "empty"). When the file is fully missing (state === "missing", as observed here — likely because the job failed before the artifact-producing step ever ran), that carve-out doesn't match, and the strict-coverage check throws instead, cascading the one bad run's failure onto the whole fleet for up to 24h.
Why it matters
This is a self-perpetuating outage: one failed run with a missing usage file blocks every subsequent scheduled run of the same workflow from ever executing its agent job, until the poisoned run ages out of the 24h window. This is currently live and actively blocking PR Sous Chef.
Suggested fix
In sumCoveredComponents (actions/setup/js/daily_aic_component_coverage.cjs, near line 152), extend the existing name === "agent" && job.conclusion === "failure" zero-cost carve-out to also accept candidateStates[0].state === "missing" (not only "empty"), since a failed job that never reached the accounting-write step is equally strong proof of no billable usage as one that wrote an empty file.
Data source
Live fleet-log spot-check (Step 2) + direct log read of run 34969377719's activation job (job 104381785440) + source read of actions/setup/js/daily_aic_component_coverage.cjs and check_daily_aic_workflow_guardrail.cjs, this cycle. Dedup-checked: #61081/#61082 are auto-generated per-run failure reports for the original poisoned run and do not mention or diagnose the cascading guardrail bug; no other open issue found via local issues snapshot grep or search_issues.
Suggested Agent: general coding agent (Node/JS, gh-aw actions internals)
Estimated Effort: Small (one-line condition change + regression test for the missing-file case in check_daily_aic_workflow_guardrail.test.cjs)
Warning
Firewall blocked 1 domain
The following domain was blocked by the firewall during workflow execution:
api.anthropic.com
To allow these domains, add them to the network.allowed list in your workflow frontmatter:
network:
allowed:
- defaults
- "api.anthropic.com"See Network Configuration for more information.
Generated by 🔬 Deep Report · claude · agent · 279.5 AIC · ⌖ 12.7 AIC · ⊞ 12.8K · ◷
- expires on Sep 17, 2026, 4:50 AM UTC-08:00
Activity