Skip to content

fix(agent-approval-check): recognise the commits this action itself pushes - #1869

Open
v0ropaev wants to merge 1 commit into
anthropics:mainfrom
v0ropaev:fix/approval-gate-sees-action-commits
Open

v0ropaev wants to merge 1 commit into
anthropics:mainfrom
v0ropaev:fix/approval-gate-sees-action-commits

Conversation

@v0ropaev

Copy link
Copy Markdown

What is wrong

agent-approval-check gates a PR on human approvals once it sees agent activity. It cannot see the commits claude-code-action itself pushes, so on the default path it posts success: No agent activity and the gate never engages.

is_agent_commit matches only the committer email against agent_emails, whose default is noreply@anthropic.com:

def is_agent_commit(commit: dict, config: AgentConfig) -> bool:
    email = get_committer_email(commit).lower()
    return email in (e.lower() for e in config.agent_emails)

noreply@anthropic.com is real, it is what Claude Code writes locally, but it is not what this action pushes as. With use_commit_signing at its default of false, src/github/operations/git-config.ts:42 runs

git config user.email "${botId}+${botName}@users.noreply.github.com"

with bot_id: "41898282" and bot_name: "claude[bot]" from action.yml, so the commit carries 41898282+claude[bot]@users.noreply.github.com. Real claude[bot] commits on GitHub carry 209825114+claude[bot]@users.noreply.github.com.

The login checks do not cover it either. The action never opens a PR: src/create-prompt/index.ts:604 hands the human a compare/...?quick_pull=1 link to click, and there is no pulls.create anywhere in src/. So the PR author is a human and is_pr_created_by_agent is False, and the committer login is not in the GraphQL commit selection at all.

Running the real module against the real addresses:

action default path (bot_id 41898282)    41898282+claude[bot]@users.noreply.github.com   -> False
claude[bot] commits in the wild          209825114+claude[bot]@users.noreply.github.com  -> False
use_commit_signing: true                 noreply@github.com                              -> False
control: the configured value            noreply@anthropic.com                           -> True

human-opened PR of claude[bot] commits -> has_agent_activity: False

What this changes

A GitHub noreply committer address is <id>+<login>@users.noreply.github.com, and it names the login. The id is not worth matching on, since bot_id is a configurable input and the App's own id differs from the default, but the login is stable and agent_logins already lists the agents. So a commit also counts when its committer address is a GitHub noreply address whose login is in agent_logins.

41898282+claude[bot]@users.noreply.github.com   -> True   (agent claude[bot])
209825114+claude[bot]@users.noreply.github.com  -> True   (agent claude[bot])
1234+claude-code[bot]@users.noreply.github.com  -> True   (agent claude-code[bot])
noreply@anthropic.com                           -> True   (agent email, unchanged)
555+alice@users.noreply.github.com              -> False
alice@example.com                               -> False
noreply@github.com                              -> False

get_detection_reason had an assert that the email is in agent_emails, which would now fire, so it reports which rule matched instead. The three places that document the input say what the login match covers.

What this does not fix

use_commit_signing: true still gets through. Those commits go to /git/commits with no committer (src/mcp/github-file-ops-server.ts:344), so GitHub stamps noreply@github.com, which every commit made through the web UI also carries. It cannot go in an allowlist without waving through humans. Catching that one needs committer { user { login } } added to GRAPHQL_PR_QUERY and carried through the normalizer. Happy to do it in this PR if you want it here rather than separately.

Test Plan

There is no test harness for agent_approval_check.py in the repo, so I drove the module directly: loaded it with importlib, built an AgentConfig with the shipped defaults, and ran is_agent_commit, get_detection_reason and check_for_agent_activity over the seven addresses above. The table is that run, before and after. The two claude[bot] addresses are not invented, they come from search/commits?q=author:claude[bot] on live repos.

python3 -m py_compile clean, prettier --check clean on the README, the action metadata and the example YAML, and bun test test/action-metadata.test.ts passes.

If you would like a tests/ file for this action I am happy to add one, but it would be the first, so I did not assume the layout.

…ushes

is_agent_commit matches a commit only when its committer email is listed in
agent_emails, whose default is noreply@anthropic.com. That is the address
Claude Code writes locally, but it is not what claude-code-action pushes as.
On the default path, use_commit_signing false, git-config.ts sets
user.email to "${botId}+${botName}@users.noreply.github.com", so the commit
lands as 41898282+claude[bot]@users.noreply.github.com, and claude[bot]
commits in the wild carry 209825114+claude[bot]@… .

Neither matches, and the login checks do not help: the action never opens
the PR, it hands the human a compare/?quick_pull=1 link, so the author is a
human and is_pr_created_by_agent is False. The gate posts
"success: No agent activity" on a PR made entirely of agent commits.

    human-opened PR of claude[bot] commits -> has_agent_activity: False

A GitHub noreply committer address names its login, and agent_logins already
lists the agents, so match on that too. The id is not worth matching on,
since bot_id is a configurable input.

    41898282+claude[bot]@users.noreply.github.com   -> True
    209825114+claude[bot]@users.noreply.github.com  -> True
    1234+claude-code[bot]@users.noreply.github.com  -> True
    noreply@anthropic.com                           -> True
    555+alice@users.noreply.github.com              -> False
    alice@example.com                               -> False

use_commit_signing: true still gets past this. GitHub stamps those commits
with committer noreply@github.com, which every web-UI commit also carries,
so it cannot go in a list. Catching it needs committer { user { login } } in
the GraphQL commit selection.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant