[audit-workflows] Daily Agentic Workflow Audit — 2026-08-26 #56143
Replies: 3 comments
|
🤖 Just dropped in to say hi! The smoke test machines have spoken: some passed, some fizzled. Stay tuned for more adventures!
|
|
🤖✨ Beep boop! The ARM64 smoke test agent zoomed through here on its little aarch64 legs! Just popping in to say: silicon architectures may differ, but our love for green checkmarks is universal. 🟩🦾 Carry on, humans (and fellow bots)!
|
|
This discussion was automatically closed because it expired on 2026-08-27T22:29:05.316Z.
|
Uh oh!
There was an error while loading. Please reload this page.
Window: 2026-08-25 22:15 UTC → 2026-08-26 21:54 UTC (24h, 410 runs, all pages collected)
Headline numbers
By engine (excl. intentional tests): pi 96.7% (89/92) · copilot 93.3% (166/178) · claude 85.7% (60/70) · codex 74.5% (38/51, worst)
📊 Trend charts
Success counts have grown roughly in line with fleet size since mid-June, and today's 89% rate sits comfortably inside the historical 81–97% band — no fleet-wide regression, but the 51-day data gap (shaded) means this is really a same-day snapshot, not a trend confirmation.
Today's 25.9M tokens is the highest single-day figure since the 06-19 reporting gap that a prior audit flagged — but ~7.1M of that (27%) was burned by five codex runs that ultimately failed (see Finding 2), so the real "useful" token spend is lower than the raw total suggests.
Findings
1.
Avengerfails 100% of the time (8/8), zero-turndriver_exit, cross-engine pattern seen in 22 runs totalEvery one of the 8
Avengerruns today died with 0 tokens/0 turns before the agent produced anything (§33008205964). The same zero-turndriver_exitsignature hit 22 runs across 13 workflows and all three of claude (9), codex (8), and copilot (5) — this is an infra/startup-level failure, not workflow logic. Notably, these runs already ran oncli_version 6bd17ee, the same commit that just landed "Retry Claude runs after AWF/MCP startup failures" (#55864) — yetAvenger(claude engine) still failed every single time, and the pattern also spans codex/copilot which that fix's title suggests it may not cover. Worth checking whether the retry logic actually reaches these workflows or their compiled.lock.ymlpredates the fix.2. Codex "runaway context rebuild" burns ~7.1M tokens across 5 guaranteed-to-fail runs
Daily Project Performance Summary Generator,The Daily Repository Chronicle,Terminal Stylist,Sub-Issue Closer, andMetrics Collector(§32924199594) each ran 35–41 invocations with a working-setrebuild_factorof ~25–26× (vs. ~6.3× median for healthy codex runs today) — full context reconstruction on nearly every turn — burning 1.1M–1.75M tokens each before still hittingdriver_exit. Combined this is ~7.1M tokens (27% of the day's fleet-wide total) spent on runs that failed anyway. A working-set health check that fails fast once rebuild_factor blows past ~2-3× the baseline would avoid most of this waste.3. Cost outlier:
Agent Job Health Monitor— 1,038 AIC / 552K tokens in one runThis single successful run (§32909810347) cost more than 2× the next most expensive run of the day and ~13× the median top-15 workflow cost. Worth a quick check on whether this is expected for a monitoring job or a runaway prompt/context issue.
4.
Code Scanning Fixer— 3 of 4 failures share a single-turn, ~25–28 min, ~15–19K token signatureClassified
agent_logic, notdriver_exit: the agent takes exactly one turn, burns a modest 15–19K tokens, runs ~25–28 minutes, then fails. That duration-vs-token-count mismatch (long wall-clock time for so few tokens) suggests a slow/hanging tool call rather than a reasoning failure — worth checking what that one turn was waiting on.5. Network friction:
smoke-copilotblocked 24% of its requests (35/147)Highest firewall block rate of the day, distantly followed by
daily-syntax-error-quality-check(12%) andcode-scanning-fixer(6%). Not causing failures today, but worth a look at whether the copilot smoke test's allowed-domains list needs updating.6. Two previously-tracked known issues appear resolved — recommend closing after verification
smoke-ci-100pct-fail(was the dominant failure cause as of 2026-07-06, 31% of all fleet failures): theSmoke CIworkflow no longer appears in the fleet at all (renamed/removed?); onlySmoke Copilotexists now and it isn't failing.token-usage-reporting-gap(fleet token totals were null/near-zero since 2026-06-19): today's token accounting is fully populated (25.9M across the fleet), so this looks fixed.Repo memory
/tmp/gh-aw/repo-memory/default/was mounted read-only for this run (confirmed on all 7 memory files; nopush_repo_memorytool was available either). The findings above are based on the existing memory (last updated 2026-07-06) plus this run's fresh log analysis, butknown-issues.json,recommendations.json,anomalies.json,workflow-trends.json,metrics-summary.json, andaudit-history.jsonlwere not updated this cycle. Please check the repo-memory mount/write-permission setup for this workflow — without it, the two stale issues noted above can't be closed and today's new findings (Avenger, codex runaway-rebuild) won't carry forward to tomorrow's audit.References:
All reactions