Skip to content

Let an agent ask for tool approval in more than one step - #74686

Merged
kaxil merged 2 commits into
mainfrom
commonai-tool-approval-repeat
Oct 14, 2026
Merged

kaxil merged 2 commits into
mainfrom
commonai-tool-approval-repeat

Conversation

@kaxil

@kaxil kaxil commented Oct 14, 2026

Copy link
Copy Markdown
Member

Stacked on #74679.

Today an AgentOperator task can pause for tool approval only once per Dag run; a second request fails the task with ToolApprovalAlreadyRequestedError. An agent decides its next call after it sees the last result, so a refund agent that approves order 1, reads the result and then asks to refund order 2 fails on the second ask.

The limit came from core: on Airflow 3.3, a repeat approval request for the same task instance keeps the first request's subject and body, so the reviewer would approve a new call while reading the old one. #73235 (in 3.4) makes a repeat request replace those details and clear the earlier answer. On Airflow 3.4 and later, this PR lets the run pause at every step that calls a marked tool. On 3.3 the rule stays, with an error message that names the upgrade.

Run end to end on Airflow main with a scripted agent that refunds three orders, one per step: the task paused three times in one try, each Required Actions request showed only its own order, each refund ran once, and the task returned all three results.

Design rationale

A stale answer can still reach a later request, so each request is numbered. Core accepts an answer without knowing which request it was for. If one reviewer approves request 1 and the agent pauses for request 2, a second reviewer who still has request 1's form open could approve refund 2 while reading refund 1. Each request now carries its number in its subject and in a hidden const form field, which the review form sends back with the answer. An answer naming an earlier request is not applied: the task asks about the waiting calls again under a new number. A REST API answer without the field is applied to the waiting request, so existing integrations keep working. In the end-to-end run, a deliberately stale answer for request 1 while request 2 was waiting produced request 3 for the same order, and nothing ran twice. A core change that ties every answer to its request revision is the proper fix for all HITL operators; this guard can go once that ships.

Why gate on Airflow 3.4 rather than detect the core behaviour? The behaviour is a property of the core version, AIRFLOW_V_3_4_PLUS is already in the common.compat floor, and #73235 was not backported to a 3.3 release.

Each resumed run gets its own run_id, <ti id>-resumed-<n>. pydantic-ai refuses a run_id that already appears in the history, so a second resume with the old fixed <ti id>-resumed failed.

Gotchas

  • A retry or a clear asks again for the calls the new attempt makes, including ones approved in an earlier attempt that then failed. The approved call has already run once, as before.
  • usage_limits counts across every pause, so a run with many gated steps can reach request_limit (50 by default) partway through.
  • The run_id XCom of a resumed run changes from <ti id>-resumed to <ti id>-resumed-<n>.

Per-tool approval moves to the stable table in stability.rst, since the once-per-run limit was the reason #74679 keeps it experimental.

Base automatically changed from commonai-stability-graduation to main October 14, 2026 07:35
On Airflow 3.4 and later, an AgentOperator run pauses for approval at
every step that calls a marked tool, instead of failing the task on its
second request. A real agent decides each gated call after it sees the
previous result, so the once-per-Dag-run rule failed it on its second
approval.

The rule existed because Airflow 3.3 keeps the first request's subject
and body on a repeat request. Airflow 3.4 replaces them and clears the
earlier answer (#73235), so a second pause shows the reviewer only its
own calls. On 3.3 the rule stays.

Core still accepts an answer without knowing which request it was for,
so each request now carries its number in a hidden form field and in
its subject. An answer that names an earlier request, from a form left
open on that request, is not applied: the task asks about the waiting
calls again.

Each resume gets its own run_id, since pydantic-ai refuses a run_id
already in the history, and keeps the transcript when the resumed run
pauses again. Per-tool approval moves to the stable table.
@kaxil
kaxil force-pushed the commonai-tool-approval-repeat branch from 474626c to 4599b4f Compare October 14, 2026 07:35
The test resumes two pauses but relied on the real gate, so it failed on
the Airflow 3.3 compat job, where a second request is refused. Patch the
gate open as the other multi-pause tests do; the decorator had been left
on the stale-answer test above it.
@kaxil
kaxil merged commit 2bf8c2e into main Oct 14, 2026
85 checks passed
@kaxil
kaxil deleted the commonai-tool-approval-repeat branch October 14, 2026 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants