Repository navigation
Add BaseManagedAgentToolset for vendor-managed AI agents - #71946
Conversation
Cloud vendors now run agents on the user's behalf -- Snowflake Cortex Agents, Amazon Bedrock AgentCore, Azure AI Foundry hosted agents, Vertex AI Agent Engine. Their reasoning loops execute on the vendor's infrastructure, so they are not something AgentOperator can run, only something an Airflow task can consult. Each provider that has added support so far arrived at a different identity scheme, request shape, and invoke verb, so a Dag author who learns one learns nothing transferable about the next. Rather than unify them behind an operator facade -- their invocation semantics genuinely differ, from sync request/response to async jobs writing to object storage to stateful sessions -- converge them at the tool boundary, where a single call with validated arguments and a serialisable result is the same shape everywhere. Provider packages subclass this contract so credentials keep flowing through their own hooks and no new connection types are needed. Errors are deliberately sorted into three buckets rather than two: what the calling model can fix by rephrasing, what is terminal, and what is transient and belongs to Airflow's task retry. Collapsing the third into either of the others is how these implementations go wrong. Durable replay is opt-in per implementation, since a managed agent may act on systems Airflow cannot observe and replaying a cached answer could skip a side effect. Also promotes the private tool-result serialiser out of HookToolset into the shared tool_definition utils so both toolsets use one implementation.
Cloud vendors now run agents on the user's behalf -- Snowflake Cortex Agents, Amazon Bedrock AgentCore, Azure AI Foundry hosted agents, Vertex AI Agent Engine. Their reasoning loops execute on the vendor's infrastructure, so they are not something AgentOperator can run, only something an Airflow task can consult. Each provider that has added support so far arrived at a different identity scheme, request shape, and invoke verb, so a Dag author who learns one learns nothing transferable about the next. What all of them do agree on is a single consultation: a question in, an answer out, once. That is the shape of a tool call, so the contract is scoped to exactly that and leaves each provider's own lifecycle operators alone. Provider packages subclass it, keeping credentials flowing through their existing hooks and requiring no new connection types. Errors are deliberately sorted into three buckets rather than two: what the calling model can fix by rephrasing, what is terminal, and what is transient and belongs to Airflow's task retry. Collapsing the third into either of the others is how these implementations go wrong. Durable replay is opt-in per implementation, since a managed agent may act on systems Airflow cannot observe and replaying a cached answer could skip a side effect. FailoverManagedAgentToolset composes interchangeable agents -- the same agent deployed on two clouds -- into one tool for active/passive failover. It is itself a BaseManagedAgentToolset, so the model sees a single tool and the policy stays deterministic rather than becoming a prompt instruction a model may ignore, and groups nest. Task-level failover remains the better choice for a standalone call; this covers the case a task boundary cannot express, where the agent is consulted mid-run and failing the task would discard the calling agent's accumulated context. Also promotes the private tool-result serialiser out of HookToolset into the shared tool_definition utils so both toolsets use one implementation.
BaseManagedAgentToolset required a description and raised on a blank one, on the reasoning that a remote agent's competence cannot be introspected and the string is what the calling model routes on. Two things were wrong with that. HookToolset, in the same package, does not require one -- it derives a description from the method name when a method has no docstring. Requiring one here while deriving one there is an inconsistency with nothing behind it, and it trusted the author's judgement on the tool name while refusing to trust it on whether a description was needed. The routing argument also described a pattern that is no longer the recommended one. A task holds a single managed agent toolset, and a failover group presents one tool, so in both cases the model is not choosing between specialists at all. The name stays required: it is the identifier the model emits, and an empty one is a bug. The description falls back to the name rendered as prose, and the docs now recommend writing one for what it actually does -- telling the model to consult the agent rather than answer from its own knowledge, and recording a scope limit the name cannot carry.
Reads better as an explicit integer comparison: `if position > 0` says "not the primary", where the bare truthiness check made the reader work out that position is an enumerate index. Named it so the two places that ask the same question read the same way, matching the explicit form used throughout airflow-core. The remaining changes are comment and docstring corrections, not behaviour. Two justifications were written for a usage pattern that is no longer the recommended one -- a single agent holding several managed agent toolsets and choosing between them. A task holds one toolset, so sequential=False is worth having for a different reason (nothing is shared between calls, unlike HookToolset's tools sharing a hook), and agent_ref's value is recording which remote identity a task resolved to rather than distinguishing between several. max_retries=1 was inherited from HookToolset without comment; it now says why one attempt is right here, given how expensive an invocation is.
…/airflow into common_ai_managed_toolset
| ctx: RunContext[Any], | ||
| tool: ToolsetTool[Any], | ||
| ) -> Any: | ||
| ref = self.agent_ref |
There was a problem hiding this comment.
call_tool resolves self.agent_ref before it calls invoke(), and on a failover group that property (line 306) joins agent_ref for every member, standbys included, so a standby whose identity comes from a missing or misconfigured connection makes the whole tool call fail before the healthy primary is tried. Since the standby usually lives on a different cloud, a broken standby connection is the likely case rather than the exotic one. I ran it against this code: group.invoke(prompt) returns the primary's answer, while group.call_tool(...) on the same group raises RuntimeError: Connection 'azure_standby' not found.
| async def invoke(self, prompt: str) -> Any: | ||
| last = len(self._members) - 1 | ||
| for position, member in enumerate(self._members): | ||
| ref = member.agent_ref |
There was a problem hiding this comment.
ref = member.agent_ref sits outside the try, and the standby lookup on line 323 sits inside the except handler but is equally unguarded, so a failure to resolve identity bypasses failover entirely: a broken primary ref raises before any member is invoked, and a broken standby ref replaces the primary's transient failure with the standby's config error without the standby ever being tried. The docs say a transient failure moves to the next member, but with a misconfigured standby a primary 503 surfaces as a connection error instead. Resolving each ref lazily, or wrapping both lookups so a failed lookup degrades to the name alone, would keep failover independent of identity resolution.
| self._tool_name = tool_name | ||
| # Same fallback as HookToolset uses for a method with no docstring. | ||
| self._description = (description or "").strip() or tool_name.replace("_", " ").capitalize() | ||
| self._timeout = timeout |
There was a problem hiding this comment.
timeout is documented as the seconds to wait for a single invocation, but nothing in the base class reads self._timeout, so the only way a subclass can honor it is to reach into a private attribute of another package's class. agent_ref and replayable are public for exactly that reason. A public timeout property, plus a docstring line saying that honoring it is the implementation's job, would give provider subclasses a supported way in.
| # Copied, not aliased: the loop in invoke() relies on the group being | ||
| # non-empty, and a caller holding the original list could otherwise empty | ||
| # it after construction. | ||
| self._members = tuple(members) |
There was a problem hiding this comment.
Inside a group, each member's tool_name, description and max_retries are inert: the group builds its single ToolDefinition from its own values and only ever calls member.invoke(), never member.get_tools(). The docs example still has to give each member a tool_name that nothing reads, so a line in the docs would stop someone tuning max_retries on a member and wondering why it does nothing.
| # Emitted on every answer so the standby-served fraction is a ratio of | ||
| # this counter, not something that has to be scanned out of XCom. | ||
| Stats.incr( | ||
| "managed_agent.served", |
There was a problem hiding this comment.
managed_agent.served only ever fires inside a failover group, so a plain BaseManagedAgentToolset, which the docs call the normal case of one toolset per task, emits no metrics at all, and the failover counter is left without a denominator. The name reads as though it covers every managed agent invocation while counting only the grouped ones. Should the base class emit it too, taking care not to double count when the group's inherited call_tool runs?
An agent's ``agent_ref`` exists only to label a call for logs and metrics, but it was resolved eagerly and unguarded, so a connection lookup failing inside it blocked the invocation it merely described. A failover group resolves every member's identity, so one misconfigured standby -- usually on a different cloud from the primary, and therefore the likely case rather than the exotic one -- failed the whole tool call before the healthy primary was ever tried. The same gap made failover itself depend on identity resolution: a broken primary ref raised before any member was invoked, and a broken standby ref replaced the primary's transient failure with the standby's configuration error without the standby being tried at all. ``timeout`` was documented but not readable from a subclass, leaving implementations to reach into another package's private attribute to honour it. ``managed_agent.served`` only ever fired inside a failover group, so the failover counter had no denominator and a lone toolset emitted nothing at all. Raised in review of apache#71946.
Add BaseManagedAgentToolset for vendor-managed AI agents Cloud vendors now run agents on the user's behalf: Amazon Bedrock AgentCore, Azure AI Foundry hosted agents, and Vertex AI Agent Engine are primary examples. Their reasoning loops execute on the vendor's infrastructure, so they are not something AgentOperator can run, only something an Airflow task can consult. Each provider that has added support so far arrived at a different identity scheme, request shape, and invoke verb, so a Dag author who learns one learns nothing transferable about the next. Rather than unify them behind an operator facade -- their invocation semantics genuinely differ, from sync request/response to async jobs writing to object storage to stateful sessions -- converge them at the tool boundary, where a single call with validated arguments and a serialisable result is the same shape everywhere. Provider packages subclass this contract so credentials keep flowing through their own hooks and no new connection types are needed. Most vendor SDKs are synchronous -- boto3 for AgentCore, the Snowflake connector for Cortex -- while some ship async clients. A subclass implements invoke() for a natively async client, or invoke_sync() for a blocking one, which the base class runs in a worker thread so a call taking minutes does not stall the calling agent's other tool calls. Implementing neither raises at construction rather than at first call. Errors are deliberately sorted into three buckets rather than two: what the calling model can fix by rephrasing, what is terminal, and what is transient and belongs to Airflow's task retry. Collapsing the third into either of the others is how these implementations go wrong. Durable replay is opt-in per implementation, since a managed agent may act on systems Airflow cannot observe and replaying a cached answer could skip a side effect. FailoverManagedAgentToolset composes interchangeable agents -- the same agent deployed on two clouds -- into one tool for active/passive failover. It is itself a BaseManagedAgentToolset, so the model sees a single tool and the policy stays deterministic rather than becoming a prompt instruction a model may ignore, and groups nest. Because a failover is a success-shaped event, it emits managed_agent.failover and managed_agent.served counters -- without them a primary that has been down for a week looks identical to a healthy one. Co-authored-by: gopidesupavan <gopidesupavan@gmail.com>
An agent's ``agent_ref`` exists only to label a call for logs and metrics, but it was resolved eagerly and unguarded, so a connection lookup failing inside it blocked the invocation it merely described. A failover group resolves every member's identity, so one misconfigured standby -- usually on a different cloud from the primary, and therefore the likely case rather than the exotic one -- failed the whole tool call before the healthy primary was ever tried. The same gap made failover itself depend on identity resolution: a broken primary ref raised before any member was invoked, and a broken standby ref replaced the primary's transient failure with the standby's configuration error without the standby being tried at all. ``timeout`` was documented but not readable from a subclass, leaving implementations to reach into another package's private attribute to honour it. ``managed_agent.served`` only ever fired inside a failover group, so the failover counter had no denominator and a lone toolset emitted nothing at all. Raised in review of apache#71946.
An agent's ``agent_ref`` exists only to label a call for logs and metrics, but it was resolved eagerly and unguarded, so a connection lookup failing inside it blocked the invocation it merely described. A failover group resolves every member's identity, so one misconfigured standby -- usually on a different cloud from the primary, and therefore the likely case rather than the exotic one -- failed the whole tool call before the healthy primary was ever tried. The same gap made failover itself depend on identity resolution: a broken primary ref raised before any member was invoked, and a broken standby ref replaced the primary's transient failure with the standby's configuration error without the standby being tried at all. ``timeout`` was documented but not readable from a subclass, leaving implementations to reach into another package's private attribute to honour it. ``managed_agent.served`` only ever fired inside a failover group, so the failover counter had no denominator and a lone toolset emitted nothing at all. Raised in review of apache#71946.
An agent's ``agent_ref`` exists only to label a call for logs and metrics, but it was resolved eagerly and unguarded, so a connection lookup failing inside it blocked the invocation it merely described. A failover group resolves every member's identity, so one misconfigured standby -- usually on a different cloud from the primary, and therefore the likely case rather than the exotic one -- failed the whole tool call before the healthy primary was ever tried. The same gap made failover itself depend on identity resolution: a broken primary ref raised before any member was invoked, and a broken standby ref replaced the primary's transient failure with the standby's configuration error without the standby being tried at all. ``timeout`` was documented but not readable from a subclass, leaving implementations to reach into another package's private attribute to honour it. ``managed_agent.served`` only ever fired inside a failover group, so the failover counter had no denominator and a lone toolset emitted nothing at all. Raised in review of apache#71946.
An agent's ``agent_ref`` exists only to label a call for logs and metrics, but it was resolved eagerly and unguarded, so a connection lookup failing inside it blocked the invocation it merely described. A failover group resolves every member's identity, so one misconfigured standby -- usually on a different cloud from the primary, and therefore the likely case rather than the exotic one -- failed the whole tool call before the healthy primary was ever tried. The same gap made failover itself depend on identity resolution: a broken primary ref raised before any member was invoked, and a broken standby ref replaced the primary's transient failure with the standby's configuration error without the standby being tried at all. ``timeout`` was documented but not readable from a subclass, leaving implementations to reach into another package's private attribute to honour it. ``managed_agent.served`` only ever fired inside a failover group, so the failover counter had no denominator and a lone toolset emitted nothing at all. Raised in review of apache#71946.
An agent's ``agent_ref`` exists only to label a call for logs and metrics, but it was resolved eagerly and unguarded, so a connection lookup failing inside it blocked the invocation it merely described. A failover group resolves every member's identity, so one misconfigured standby -- usually on a different cloud from the primary, and therefore the likely case rather than the exotic one -- failed the whole tool call before the healthy primary was ever tried. The same gap made failover itself depend on identity resolution: a broken primary ref raised before any member was invoked, and a broken standby ref replaced the primary's transient failure with the standby's configuration error without the standby being tried at all. ``timeout`` was documented but not readable from a subclass, leaving implementations to reach into another package's private attribute to honour it. ``managed_agent.served`` only ever fired inside a failover group, so the failover counter had no denominator and a lone toolset emitted nothing at all. Raised in review of apache#71946.
An agent's ``agent_ref`` exists only to label a call for logs and metrics, but it was resolved eagerly and unguarded, so a connection lookup failing inside it blocked the invocation it merely described. A failover group resolves every member's identity, so one misconfigured standby -- usually on a different cloud from the primary, and therefore the likely case rather than the exotic one -- failed the whole tool call before the healthy primary was ever tried. The same gap made failover itself depend on identity resolution: a broken primary ref raised before any member was invoked, and a broken standby ref replaced the primary's transient failure with the standby's configuration error without the standby being tried at all. ``timeout`` was documented but not readable from a subclass, leaving implementations to reach into another package's private attribute to honour it. ``managed_agent.served`` only ever fired inside a failover group, so the failover counter had no denominator and a lone toolset emitted nothing at all. Raised in review of apache#71946.
An agent's ``agent_ref`` exists only to label a call for logs and metrics, but it was resolved eagerly and unguarded, so a connection lookup failing inside it blocked the invocation it merely described. A failover group resolves every member's identity, so one misconfigured standby -- usually on a different cloud from the primary, and therefore the likely case rather than the exotic one -- failed the whole tool call before the healthy primary was ever tried. The same gap made failover itself depend on identity resolution: a broken primary ref raised before any member was invoked, and a broken standby ref replaced the primary's transient failure with the standby's configuration error without the standby being tried at all. ``timeout`` was documented but not readable from a subclass, leaving implementations to reach into another package's private attribute to honour it. ``managed_agent.served`` only ever fired inside a failover group, so the failover counter had no denominator and a lone toolset emitted nothing at all. Raised in review of apache#71946.
An agent's ``agent_ref`` exists only to label a call for logs and metrics, but it was resolved eagerly and unguarded, so a connection lookup failing inside it blocked the invocation it merely described. A failover group resolves every member's identity, so one misconfigured standby -- usually on a different cloud from the primary, and therefore the likely case rather than the exotic one -- failed the whole tool call before the healthy primary was ever tried. The same gap made failover itself depend on identity resolution: a broken primary ref raised before any member was invoked, and a broken standby ref replaced the primary's transient failure with the standby's configuration error without the standby being tried at all. ``timeout`` was documented but not readable from a subclass, leaving implementations to reach into another package's private attribute to honour it. ``managed_agent.served`` only ever fired inside a failover group, so the failover counter had no denominator and a lone toolset emitted nothing at all. Raised in review of apache#71946.
An agent's ``agent_ref`` exists only to label a call for logs and metrics, but it was resolved eagerly and unguarded, so a connection lookup failing inside it blocked the invocation it merely described. A failover group resolves every member's identity, so one misconfigured standby -- usually on a different cloud from the primary, and therefore the likely case rather than the exotic one -- failed the whole tool call before the healthy primary was ever tried. The same gap made failover itself depend on identity resolution: a broken primary ref raised before any member was invoked, and a broken standby ref replaced the primary's transient failure with the standby's configuration error without the standby being tried at all. ``timeout`` was documented but not readable from a subclass, leaving implementations to reach into another package's private attribute to honour it. ``managed_agent.served`` only ever fired inside a failover group, so the failover counter had no denominator and a lone toolset emitted nothing at all. Raised in review of apache#71946.
An agent's ``agent_ref`` exists only to label a call for logs and metrics, but it was resolved eagerly and unguarded, so a connection lookup failing inside it blocked the invocation it merely described. A failover group resolves every member's identity, so one misconfigured standby -- usually on a different cloud from the primary, and therefore the likely case rather than the exotic one -- failed the whole tool call before the healthy primary was ever tried. The same gap made failover itself depend on identity resolution: a broken primary ref raised before any member was invoked, and a broken standby ref replaced the primary's transient failure with the standby's configuration error without the standby being tried at all. ``timeout`` was documented but not readable from a subclass, leaving implementations to reach into another package's private attribute to honour it. ``managed_agent.served`` only ever fired inside a failover group, so the failover counter had no denominator and a lone toolset emitted nothing at all. Raised in review of apache#71946.
Initial PR to create a common interface for invoking Managed Agents across clouds, which would enable:
The common interface is an abstraction, which would then be implemented across Managed Agent implementations for individual clouds including AWS Bedrock, Azure Foundry, and Google Vertex.
Just in case it wasn't clear above, this is intended to be part of the Common AI provider.
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Sonnet 5)
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.