Repository navigation
feat(ci): upstream-compat schedule + CI budget record (DEC-082) - #59
Merged
Merged
Conversation
Upstream A0 is the only thing anyone actually runs, and nothing tested it. The
reusable workflow defaults a0_image to the private fork and none of the four
plugins overrode it, so every PR proved the plugin worked on the fork while the
target that matters went unverified. Coverage pointed at the wrong image.
upstream-compat re-runs the four suites when UPSTREAM moves — something no PR can
catch, because nothing in the repo changed. Gated on the manifest digest actually
moving: measured, upstream ships ~1.5x/week against a 42-minute fan-out, so
digest-gating costs ~295 runner-min/month instead of ~1,275 for the same <=24h
detection. Runs pin the resolved digest, not :latest, so a failure is reproducible
against the exact image that broke it.
Deliberately unlike the nightly just deleted in DEC-079: that opened a PR every
night regardless and then failed silently behind an approval gate for three weeks.
This opens no PRs, commits nothing, and reports one issue that auto-closes when
green — so an open issue always means act. Three guards keep it honest: the probe
hard-fails rather than shrugging, an empty matrix is an error not a vacuous pass,
and the report job asserts the matrix ran if the probe asked for it.
CI budget record lives in a long-lived per-repo issue upserted in place, NOT a
committed file — a committed record produces a commit per run whose only delta is
a timestamp, which is exactly the churn that got devkit-sync deleted.
Worth knowing: runs/{id}/timing returns total_ms 0 on every run for these public
repos, so a budget built on it would report 0 min forever and look like it worked.
It is deliberately unused and the record says so on its face.
This was referenced Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes C8. Scoped to the four plugins we are working on.
The finding that matters
The reusable workflow defaults
a0_imageto the private fork, and none of the four plugins overrides it. So every PR has been proving the plugin works on the fork — while upstream, the only thing anyone actually runs, was verified nowhere.That is not a coverage gap so much as coverage aimed at the wrong image.
Two changes, deliberately separate
1. The PR gate should test what customers run. Each plugin gets
a0_image: docker.io/agent0ai/agent-zero:latestin its own.devkit.yml— a per-repo override rather than flipping the devkit default, so this touches only our four. A break is then caught before merge, by the author, with the diff in front of them. (Separate PRs, one per repo.)2.
upstream-compatcatches upstream moving underneath us — which no PR can catch, because nothing in the repo changed.Why this schedule is not the one we just deleted
DEC-079 removed a nightly that opened a PR every night regardless of need and then failed silently behind an approval gate for three weeks. This one is different in the ways that caused that failure:
Measured, not assumed: upstream ships ~1.5×/week, full fan-out is 42 min. Digest-gating gets the same ≤24h detection for ~76% less.
Three structural guards: the probe hard-fails rather than shrugging, an empty matrix is an error not a vacuous pass, and the report job asserts the matrix ran if the probe asked for it.
Runs pin the resolved digest, not
:latest— so a failure is reproducible against the exact image that broke it, not whatever:latestpoints at by the time someone looks.CI budget
Record lives in a long-lived per-repo issue, upserted in place — not a committed file. A committed record means a commit per run whose only delta is a timestamp, which is precisely the churn that got devkit-sync deleted.
A trap worth knowing:
runs/{id}/timingreturnstotal_ms: 0on every run for these public repos. A budget built on it would report0 minforever and look like it worked. It is deliberately unused, and the record says so on its face rather than presenting a confident zero.Not yet proven
The workflows have never executed on Actions. Highest-risk untested detail is the
plugin/checkout and/pluginmount path inupstream-compat.yml— it runs in the devkit's workspace, not the consumer's, so that path differs from the provenplugin-e2e.ymlbody. I will exercise it viaworkflow_dispatchonce merged, before claiming it works.