Repository navigation
feat(#357): create agent commits through the GitHub API so they are signed - #1675
Conversation
Functional tests are runningAuthorization passed for this commit. See the Functional Tests workflow for results. |
Functional tests did not runFunctional tests run automatically for org/repo members and collaborators on pull requests. For other contributors, a maintainer must add the |
PR Summary by QodoPublish GitHub agent commits as signed commits through the API
AI Description
Diagram
High-Level Assessment
Files changed (24)
|
Code Review by Qodo
1.
|
|
/fs-review |
|
🤖 Finished Review · ✅ Success · Started 1:07 PM UTC · Completed 1:23 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5-5 · Effort: high · Cost: $3.83 |
|
Risk Assessment: elevated (3/5) DetailsTier 1 is about 2.75, Tier 2 about 3.0 and Tier 3 about 3.0, so the composite is about 2.9 and rounds to 3. The PR grew from about 2944 to 3290 changed lines across 26 files, is breaking with no opt-in or rollback path, and touches 23 protected scripts plus CI; the prior score of 3 is preserved. Previous runRisk Assessment: elevated (3/5) DetailsThe PR is large (2944 lines) and breaking, touches protected scripts and CI, has no opt-in or rollback path, edits heavily churned files, and the issue has unresolved design discussion; composite about 3.1 rounds to 3. |
ReviewFindingsMedium
Low
Next steps:
Previous runReviewFindingsHigh
Medium
Low
Next steps:
|
…view findings Follow-up to the signed publish, from the review of fullsend-ai#1675: - The trailer step runs outside the checkout, so a repo-local trailer.*.cmd set in the sandbox cannot execute on the runner; a test plants one. - A commit that is not bot-authored loses every Authored-by line before the real one is added, and post-fix and the fix-history-rewrite skill read the trailer only from the App bot's own commits (exact noreply email) carrying exactly one, so a crafted trailer cannot be laundered through a rebase or another bot's commit. - A commit GitHub did not sign fails the publish when the token is an App installation token; other tokens keep the warning, since they never get a signature. - The fix skill's redo/reset range is computed by a tested helper, scripts/authorized-range.sh, instead of prose. - Tests for the PR-body trailer strip and for submodule entries; push globals scoped with local; the Signed commits docs trimmed as asked and the blob claim narrowed to what the code does. fullsend counts prior fix commits by git author name, which this change makes always zero; the companion fullsend PR counts them by the trailer and must land first. Signed-off-by: Gwenneg Lepage <gwenneg@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Review findings addressed in a1b611b:
|
Agent commits published through the GitHub API (fullsend-ai/agents#1675) are authored by the App's bot user and keep the fix agent's identity in an "Authored-by: fullsend-fix <...>" trailer, so a count by git author name alone sees none of them and the fix iteration cap never engages. Count a commit as a prior fix when its author name is fullsend-fix or a line of its message starts with that trailer. Both forms count, so this is safe to land before the agents change. Signed-off-by: Gwenneg Lepage <gwenneg@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Agent commits published through the GitHub API (fullsend-ai/agents#1675) are authored by the App's bot user and keep the fix agent's identity in an "Authored-by: fullsend-fix <...>" trailer, so a count by git author name alone sees none of them and the fix iteration cap never engages. Count a commit as a prior fix when its author name is fullsend-fix or a line of its message starts with that trailer. Both forms count, so this is safe to land before the agents change. Signed-off-by: Gwenneg Lepage <gwenneg@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The filter accepted the trailer from any author, so anyone who can push to the PR branch could reach the cap and block /fs-fix. Pass the fix job's GIT_BOT_EMAIL as --arg bot_email and count the trailer only when the commit author email equals it, the rule authorized-range.sh applies in fullsend-ai/agents#1675. Signed-off-by: Gwenneg Lepage <gwenneg@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
/fs-review |
|
Update: the API publish path is now behind a switch, and the switch ships off Short version: merging this PR no longer changes how any repository pushes. The new code only runs where someone turns it on. What changed in 4e34caf
Why merging is safe now
How it is tested
Proposed rollout
Known follow-ups and tech debt
🤖 Generated by Claude Code |
|
I hope this will make it easier. As stated above, I only tested on a scratch repo so far. But I maintain or own 14 Red Hat repos currently onboarded to Fullsend and ready to test signed commits, so I'll have plenty of occasions to test this once it's merged. I don't think splitting the PR into smaller ones would really make sense here, but I can still do it if that helps. |
… they are signed Commits built by git in the sandbox are pushed unsigned, and a repository whose rules require signed commits rejects every agent PR. GitHub signs the commits it creates itself for a GitHub App installation token, so post-code and post-fix now rebuild each commit that origin does not have yet through the Git Data API (blobs, tree, commit) with the push token they already hold, move the branch ref with the updateRefs mutation, and move the local branch onto the commits GitHub created. GitLab keeps git push. GitHub makes the App's bot user the author of every rebuilt commit, so the identity the sandbox committed with is kept in an `Authored-by: name <email>` trailer, trusted only on bot-authored commits; post-fix and the fix-history-rewrite skill read it so redo and reset still tell fix-agent commits from code-agent and human ones. The ref update is a compare-and-swap anchored to the remote-tracking ref, which keeps the --force-with-lease semantics of git push. Merges are rebuilt against the parent with the smaller diff and blobs GitHub already holds are not uploaded again, to stay under the secondary rate limit. The API's rejection wording feeds the existing retry and failure categorizer. The approach was prototyped by Hector Martinez Lopez (@rh-hemartin) in rh-hemartin-fullsendai/standalone-fullsend (gh-app-commit.py) and proposed in fullsend-ai#357. On GitHub, with the API publish enabled, agent commits are authored by the App's bot user and signed by GitHub, with the sandbox identity in an `Authored-by` trailer. A signature made inside the sandbox (gitsign or SSH signing configured through the harness env) does not survive publication. Tooling that matched agent commits by git author name must read the trailer instead. GitLab is unchanged. Closes fullsend-ai#357 Co-authored-by: rh-hemartin <87312991+rh-hemartin@users.noreply.github.com> Signed-off-by: Gwenneg Lepage <gwenneg@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…view findings Follow-up to the signed publish, from the review of fullsend-ai#1675: - The trailer step runs outside the checkout, so a repo-local trailer.*.cmd set in the sandbox cannot execute on the runner; a test plants one. - A commit that is not bot-authored loses every Authored-by line before the real one is added, and post-fix and the fix-history-rewrite skill read the trailer only from the App bot's own commits (exact noreply email) carrying exactly one, so a crafted trailer cannot be laundered through a rebase or another bot's commit. - A commit GitHub did not sign fails the publish when the token is an App installation token; other tokens keep the warning, since they never get a signature. - The fix skill's redo/reset range is computed by a tested helper, scripts/authorized-range.sh, instead of prose. - Tests for the PR-body trailer strip and for submodule entries; push globals scoped with local; the Signed commits docs trimmed as asked and the blob claim narrowed to what the code does. fullsend counts prior fix commits by git author name, which this change makes always zero; the companion fullsend PR counts them by the trailer and must land first. Signed-off-by: Gwenneg Lepage <gwenneg@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… and the sign-off helpers The publisher kept an existing Authored-by trailer on any commit whose author name ended in [bot], so a forged trailer on another bot's commit survived publication and the App then became its author. It now keeps the trailer only when the author is the App bot itself, name and GIT_BOT_EMAIL, the rule commit_author_identity applies. A human commit republished through the API is authored by the App bot too, which the Signed-off-by helpers took for an agent commit and stripped its sign-off on the next merge-base scan. commit_author_identity moves into precommit-gate.lib.sh and signoff_is_bot_commit compares the author it returns; the filter-branch filter rewrites the commits the walk selected, named through GIT_COMMIT. Signed-off-by: Gwenneg Lepage <gwenneg@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… procedure
Step 5 still derived the rewrite base from ${oldest}, which step 4 no
longer defines since it runs scripts/authorized-range.sh. The code-agent
sentence now points at the identity rule in SKILL.md.
Signed-off-by: Gwenneg Lepage <gwenneg@gmail.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… the API A human commit the post-script already published is authored by the App bot with an Authored-by trailer. Rebase it onto an advanced target and run post-fix with a human rebase request: the preservation check must find it by patch-id and the trailer author, and the republished commit must keep the trailer. commit_as_published takes an optional trailer email so the trailer can name a human address. Signed-off-by: Gwenneg Lepage <gwenneg@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d helper The lost-published-code-agent and spoofed-trailer refusals are the same flow as run_redo_refused_test with a different seed and message, so they become seed functions. The post-code PR-body test exercised its own copy of the body pipeline, not the script; a grep on the script for the Authored-by sed line gives the same guarantee. Signed-off-by: Gwenneg Lepage <gwenneg@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The ownership rule is stated once in commit_author_identity and once in SKILL.md; the other copies become pointers or go. Bundles regenerated. Signed-off-by: Gwenneg Lepage <gwenneg@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… helper The four `A && B || C` assertions in fix-authorized-range-test.sh fail the pre-commit shellcheck hook (SC2015), which the required `test` check runs. An `expect` helper written as an explicit if/else replaces them; the cases, repositories and expected output are unchanged. Signed-off-by: Gwenneg Lepage <gwenneg@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…_COMMITS, off by default forge_push_branch takes the Git Data API path only when GITHUB_SIGNED_COMMITS is "true"; with any other value, or unset, it runs the same `git push -u origin BRANCH` (and `--force-with-lease` retry) as before this PR. harness/code.yaml and harness/fix.yaml set it to "false" in their GitHub blocks, so merging this PR changes nothing at runtime; a repository opts in through a `base:` overlay and the fleet default flips in a later one-line change. eval/harness/code.yaml and eval/harness/fix.yaml turn the switch on for the functional evals, registered in the root config.yaml like eval/harness/review.yaml, so every PR keeps exercising the API path end to end. Tests: the existing GitHub runs of the post-code, post-fix and library suites export the switch on, and post-code now asserts the remote tip was rebuilt by the (fake) API; the same suites run the plain-push path with the switch unset, "false" and "1", including git's non-fast-forward wording, the --force-with-lease pass-through, and the post-fix append, redo and rebase publishes, with any Git Data API call failing the run; yq checks pin the harness default, the eval overlays and their registration; the bundles are checked for the switch. Signed-off-by: Gwenneg Lepage <gwenneg@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
3c4496e to
4e34caf
Compare
|
Thanks for the additional effort, the gate makes it easier. |
#5075) Temporarily pins the dispatch reusable workflow to a non-release build so the agents-repo fallback resolves fullsend-ai/agents@main, which now includes fullsend-ai/agents#1675 (signed commits via the GitHub Git Data API). Revert to the v0.49.0 pin once testing is done. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Adds local .fullsend/ overlays that base on the merged fullsend-ai/agents#1675 harnesses (pinned by commit SHA + sha256) and flip GITHUB_SIGNED_COMMITS to "true", since the feature ships off by default. Lets the code/fix agents actually exercise the new Git Data API publish path on this repo. Revert once testing is done. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
|
Here's our first PR created from |
Summary
Closes #357.
On GitHub, with
GITHUB_SIGNED_COMMITSset to"true", post-code and post-fix no longergit pushthe commits git built in the sandbox. A ruleset that requires signed commits rejects those, because no signing key can exist in the sandbox. Instead the post-scripts rebuild each commit origin lacks through the Git Data API with the App installation token they already hold (blobs, tree, commit), move the branch ref with theupdateRefsmutation, and move the local branch onto the commits GitHub created. GitHub signs every commit it creates this way, so agent commits show as Verified and passrequired_signaturesrulesets. GitLab keepsgit push.The approach was prototyped by @rh-hemartin in rh-hemartin-fullsendai/standalone-fullsend (gh-app-commit.py).
What else had to change
Authored-by: name <email>trailer.commit_author_identityinprecommit-gate.lib.shreads it only from the App bot's own commits, exact noreply email and a single trailer; post-fix's history checks and theSigned-off-byhelpers of both post-scripts go through it, so redo and reset still tell fix-agent commits from code-agent and human ones, a human commit republished under the bot keeps its sign-off, and a crafted trailer changes nothing. Thefix-history-rewriteskill runs in the sandbox without the post-script libraries, so its newscripts/authorized-range.shapplies the same rule there. A commit the App bot did not author loses any existingAuthored-byline before publication, and the trailer step runs outside the checkout so a repo-localtrailer.*.cmdcannot execute on the runner. The PR-body fallback strips the trailer.--force-with-leasesemantics. The ref update is a compare-and-swap (updateRefswithbeforeOid) anchored to the remote-tracking ref, which is exactly whatgit push --force-with-leasechecks. The API's "not a fast forward" wording feeds the existing retry and the push failure categorizer.docs/code.md.errors[]detail reaches the failure comment. A commit GitHub did not sign fails the publish under an App installation token and is published with a warning under any other token, which never gets a signature.Rollout
The switch ships off:
GITHUB_SIGNED_COMMITS: "false"in the GitHub blocks ofharness/code.yamlandharness/fix.yaml, so merging this PR changes nothing at runtime. With the switch off,forge_push_branchruns the samegit push(and--force-with-leaseretry) as before. The functional evals turn it on througheval/harness/code.yamlandeval/harness/fix.yaml, so every PR keeps exercising the API path. A repository opts in (and, once the default flips, out) with abase:overlay that sets the variable, the documented override path (ADR 0080). Flipping the fleet default is a later one-line PR, and so is reverting it.When the switch is on, agent commits are authored by the App's bot user and signed by GitHub, with the sandbox identity in an
Authored-bytrailer. A signature made inside the sandbox (gitsign or SSH signing configured through the harness env) does not survive publication. Tooling that matched agent commits by git author name must read the trailer instead. GitLab is unchanged either way.Dependency: fullsend's dispatch counts prior fix commits by git author name (
reusable-dispatch.yml), which this change makes always zero, so the fix iteration cap would stop working. The companion PR fullsend-ai/fullsend#8425 counts them by theAuthored-bytrailer as well; it merged on 2026-10-21.Verification
"false"and"1", git's non-fast-forward wording and the--force-with-leasepass-through are checked, and post-code and post-fix run their append, redo and rebase publishes through a plaingit pushwith any Git Data API call failing the run. yq checks pin the harness default, the eval overlays and their registration.scripts/github-signed-push-test.sh(new, 25 scenarios against a bare repo withtest-fixtures/fake-gh-git-api.shstanding in for the API),signoff-strip-test.sh(a published human commit keeps its sign-off),post-code-test.sh,post-fix-test.sh(three new history-rewrite scenarios on API-published commits),post-failure-report-test.sh,validate-code-output-test.sh: all pass on Ubuntu, apart from the pre-existingyq-dependent harness check.make check-bundle,make lint(skillsaw strict),pre-commit run --all-files,hack/lint-agent-docs: clean.post-code.shandpost-fix.shfrom this branch, run on Linux with the lab App's token against the same scratch repository: post-code published two commits and opened PR #2; post-fix appended a fix commit; a second post-fix run redid it (reset past the published fix commit, recognised through its trailer, lease push). All three commits on that PR read backverified: true.verified: false, reason: unsignedon the same scratch repository; the library warns and publishes it, as documented.verified: true, reason: valid, authormq-bypass-lab-bot[bot], committer GitHub; theAuthored-bytrailer survived a---line in the body; a non-fast-forward push was refused and the--force-with-leasecompare-and-swap throughupdateRefsthen moved the branch. Repeated with an activerequired_signaturesruleset on the check branches: the API-published commits were accepted, and a git-built commit pushed withgit pushand the same App token was rejected (GH013 … Commits must have verified signatures). Branches signed-push-check-1791979153 and signed-push-check-1791982020 are left there for inspection.Known follow-ups (not in this PR)
history_rewrite_preserves_remote_human_commitsinstead of per remote commit.🤖 Generated with Claude Code