Skip to content

Fix common.ai managed agent failover when an identity cannot resolve - #72786

Merged
Lee-W merged 3 commits into
apache:mainfrom
astronomer:common-ai-managed-agent-review-followup
Oct 8, 2026
Merged

Lee-W merged 3 commits into
apache:mainfrom
astronomer:common-ai-managed-agent-review-followup

Conversation

@Lee-W

@Lee-W Lee-W commented Sep 9, 2026 •

Copy link
Copy Markdown
Member

Why

Follow-up to review comments on #71946.

When identity resolution fails, a warning is logged and the invocation still goes out, tagged with the literal platform unknown. An operator watching the counters has no way to interpret that bucket, so it looks like a broken vendor name rather than a pointer back to the task log.

What

  • docs/toolsets/managed_agent.rst: explain that platform can be "unknown" (connection lookup failed, or a bug in the subclass's agent_ref), that the call proceeds anyway, and that the task log warning at the same timestamp says why.
  • metrics_template.yaml: reword the managed_agent.served description to say the same thing.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5)

Generated-by: Claude Code (Opus 5) following the guidelines

🤖 Generated with Claude Code

Comment thread providers/common/ai/docs/toolsets.rst Outdated
Comment thread providers/common/ai/docs/toolsets.rst Outdated
Comment thread providers/common/ai/tests/unit/common/ai/toolsets/test_managed_agent.py Outdated
Lee-W added a commit to astronomer/airflow that referenced this pull request Sep 10, 2026
Incrementing after the call returned made ``managed_agent.invoked`` a
count of answers, which is a misleading denominator: a total outage still
increments ``managed_agent.failover`` while ``invoked`` stays flat, so the
documented ratio climbs fastest exactly when the group has stopped
answering, and there is nothing to divide by when nothing succeeds at all.
Counting attempts also matches what the name says.

A failover group never consumes its own ``timeout``, so accepting one and
silently dropping it left the author to discover the gap from a note they
may never read. The group now rejects it and points at the members,
alongside the validation it already does for too few members.

The metrics table the docs send readers to still described two counters
and listed the wrong tags for the two it had.

Raised in review of apache#72786.
@Lee-W
Lee-W force-pushed the common-ai-managed-agent-review-followup branch from a28ba4c to 307b2f2 Compare September 10, 2026 03:59
Lee-W added a commit to astronomer/airflow that referenced this pull request Sep 11, 2026
Incrementing after the call returned made ``managed_agent.invoked`` a
count of answers, which is a misleading denominator: a total outage still
increments ``managed_agent.failover`` while ``invoked`` stays flat, so the
documented ratio climbs fastest exactly when the group has stopped
answering, and there is nothing to divide by when nothing succeeds at all.
Counting attempts also matches what the name says.

A failover group never consumes its own ``timeout``, so accepting one and
silently dropping it left the author to discover the gap from a note they
may never read. The group now rejects it and points at the members,
alongside the validation it already does for too few members.

The metrics table the docs send readers to still described two counters
and listed the wrong tags for the two it had.

Raised in review of apache#72786.
@Lee-W
Lee-W force-pushed the common-ai-managed-agent-review-followup branch from 307b2f2 to 9f0c538 Compare September 11, 2026 10:11
Lee-W added a commit to astronomer/airflow that referenced this pull request Sep 16, 2026
Incrementing after the call returned made ``managed_agent.invoked`` a
count of answers, which is a misleading denominator: a total outage still
increments ``managed_agent.failover`` while ``invoked`` stays flat, so the
documented ratio climbs fastest exactly when the group has stopped
answering, and there is nothing to divide by when nothing succeeds at all.
Counting attempts also matches what the name says.

A failover group never consumes its own ``timeout``, so accepting one and
silently dropping it left the author to discover the gap from a note they
may never read. The group now rejects it and points at the members,
alongside the validation it already does for too few members.

The metrics table the docs send readers to still described two counters
and listed the wrong tags for the two it had.

Raised in review of apache#72786.
@Lee-W
Lee-W force-pushed the common-ai-managed-agent-review-followup branch from 9f0c538 to 896743b Compare September 16, 2026 13:42
Lee-W added a commit to astronomer/airflow that referenced this pull request Sep 17, 2026
Incrementing after the call returned made ``managed_agent.invoked`` a
count of answers, which is a misleading denominator: a total outage still
increments ``managed_agent.failover`` while ``invoked`` stays flat, so the
documented ratio climbs fastest exactly when the group has stopped
answering, and there is nothing to divide by when nothing succeeds at all.
Counting attempts also matches what the name says.

A failover group never consumes its own ``timeout``, so accepting one and
silently dropping it left the author to discover the gap from a note they
may never read. The group now rejects it and points at the members,
alongside the validation it already does for too few members.

The metrics table the docs send readers to still described two counters
and listed the wrong tags for the two it had.

Raised in review of apache#72786.
@Lee-W
Lee-W force-pushed the common-ai-managed-agent-review-followup branch from 896743b to 70f8fab Compare September 17, 2026 07:15
Lee-W added a commit to astronomer/airflow that referenced this pull request Sep 17, 2026
Incrementing after the call returned made ``managed_agent.invoked`` a
count of answers, which is a misleading denominator: a total outage still
increments ``managed_agent.failover`` while ``invoked`` stays flat, so the
documented ratio climbs fastest exactly when the group has stopped
answering, and there is nothing to divide by when nothing succeeds at all.
Counting attempts also matches what the name says.

A failover group never consumes its own ``timeout``, so accepting one and
silently dropping it left the author to discover the gap from a note they
may never read. The group now rejects it and points at the members,
alongside the validation it already does for too few members.

The metrics table the docs send readers to still described two counters
and listed the wrong tags for the two it had.

Raised in review of apache#72786.
@Lee-W
Lee-W force-pushed the common-ai-managed-agent-review-followup branch from 70f8fab to e32f86d Compare September 17, 2026 07:56
Lee-W added a commit to astronomer/airflow that referenced this pull request Sep 17, 2026
Incrementing after the call returned made ``managed_agent.invoked`` a
count of answers, which is a misleading denominator: a total outage still
increments ``managed_agent.failover`` while ``invoked`` stays flat, so the
documented ratio climbs fastest exactly when the group has stopped
answering, and there is nothing to divide by when nothing succeeds at all.
Counting attempts also matches what the name says.

A failover group never consumes its own ``timeout``, so accepting one and
silently dropping it left the author to discover the gap from a note they
may never read. The group now rejects it and points at the members,
alongside the validation it already does for too few members.

The metrics table the docs send readers to still described two counters
and listed the wrong tags for the two it had.

Raised in review of apache#72786.
@Lee-W
Lee-W force-pushed the common-ai-managed-agent-review-followup branch from e32f86d to f5a1c02 Compare September 17, 2026 08:08
Lee-W added a commit to astronomer/airflow that referenced this pull request Sep 17, 2026
Incrementing after the call returned made ``managed_agent.invoked`` a
count of answers, which is a misleading denominator: a total outage still
increments ``managed_agent.failover`` while ``invoked`` stays flat, so the
documented ratio climbs fastest exactly when the group has stopped
answering, and there is nothing to divide by when nothing succeeds at all.
Counting attempts also matches what the name says.

A failover group never consumes its own ``timeout``, so accepting one and
silently dropping it left the author to discover the gap from a note they
may never read. The group now rejects it and points at the members,
alongside the validation it already does for too few members.

The metrics table the docs send readers to still described two counters
and listed the wrong tags for the two it had.

Raised in review of apache#72786.
@Lee-W
Lee-W force-pushed the common-ai-managed-agent-review-followup branch from f5a1c02 to 30363e2 Compare September 17, 2026 08:14
Lee-W added a commit to astronomer/airflow that referenced this pull request Sep 17, 2026
Incrementing after the call returned made ``managed_agent.invoked`` a
count of answers, which is a misleading denominator: a total outage still
increments ``managed_agent.failover`` while ``invoked`` stays flat, so the
documented ratio climbs fastest exactly when the group has stopped
answering, and there is nothing to divide by when nothing succeeds at all.
Counting attempts also matches what the name says.

A failover group never consumes its own ``timeout``, so accepting one and
silently dropping it left the author to discover the gap from a note they
may never read. The group now rejects it and points at the members,
alongside the validation it already does for too few members.

The metrics table the docs send readers to still described two counters
and listed the wrong tags for the two it had.

Raised in review of apache#72786.
@Lee-W
Lee-W force-pushed the common-ai-managed-agent-review-followup branch from 30363e2 to a06c0e9 Compare September 17, 2026 08:44
@Lee-W
Lee-W marked this pull request as ready for review September 17, 2026 08:54
Lee-W added a commit to astronomer/airflow that referenced this pull request Sep 18, 2026
Incrementing after the call returned made ``managed_agent.invoked`` a
count of answers, which is a misleading denominator: a total outage still
increments ``managed_agent.failover`` while ``invoked`` stays flat, so the
documented ratio climbs fastest exactly when the group has stopped
answering, and there is nothing to divide by when nothing succeeds at all.
Counting attempts also matches what the name says.

A failover group never consumes its own ``timeout``, so accepting one and
silently dropping it left the author to discover the gap from a note they
may never read. The group now rejects it and points at the members,
alongside the validation it already does for too few members.

The metrics table the docs send readers to still described two counters
and listed the wrong tags for the two it had.

Raised in review of apache#72786.
@Lee-W
Lee-W force-pushed the common-ai-managed-agent-review-followup branch from 3ded16f to 133d869 Compare September 18, 2026 09:18
Comment thread providers/common/ai/docs/toolsets.rst Outdated
Comment thread providers/common/ai/src/airflow/providers/common/ai/toolsets/managed_agent.py Outdated
Comment thread providers/common/ai/src/airflow/providers/common/ai/toolsets/managed_agent.py Outdated
Comment thread providers/common/ai/tests/unit/common/ai/toolsets/test_managed_agent.py Outdated
Lee-W added a commit to astronomer/airflow that referenced this pull request Sep 24, 2026
Incrementing after the call returned made ``managed_agent.invoked`` a
count of answers, which is a misleading denominator: a total outage still
increments ``managed_agent.failover`` while ``invoked`` stays flat, so the
documented ratio climbs fastest exactly when the group has stopped
answering, and there is nothing to divide by when nothing succeeds at all.
Counting attempts also matches what the name says.

A failover group never consumes its own ``timeout``, so accepting one and
silently dropping it left the author to discover the gap from a note they
may never read. The group now rejects it and points at the members,
alongside the validation it already does for too few members.

The metrics table the docs send readers to still described two counters
and listed the wrong tags for the two it had.

Raised in review of apache#72786.
@Lee-W
Lee-W force-pushed the common-ai-managed-agent-review-followup branch from 133d869 to 52bf53c Compare September 24, 2026 06:22
@Lee-W
Lee-W requested a review from kaxil September 24, 2026 06:43
Lee-W added a commit to astronomer/airflow that referenced this pull request Sep 30, 2026
Incrementing after the call returned made ``managed_agent.invoked`` a
count of answers, which is a misleading denominator: a total outage still
increments ``managed_agent.failover`` while ``invoked`` stays flat, so the
documented ratio climbs fastest exactly when the group has stopped
answering, and there is nothing to divide by when nothing succeeds at all.
Counting attempts also matches what the name says.

A failover group never consumes its own ``timeout``, so accepting one and
silently dropping it left the author to discover the gap from a note they
may never read. The group now rejects it and points at the members,
alongside the validation it already does for too few members.

The metrics table the docs send readers to still described two counters
and listed the wrong tags for the two it had.

Raised in review of apache#72786.
@Lee-W
Lee-W force-pushed the common-ai-managed-agent-review-followup branch 3 times, most recently from a4e5d4b to 3f2d2f2 Compare October 4, 2026 14:09
jason810496
jason810496 approved these changes Oct 8, 2026 •
Lee-W added 2 commits October 8, 2026 09:16
An operator watching these counters has no way to interpret an "unknown"
platform bucket on their dashboard, and it is precisely the case this
change introduced: identity resolution failed, a warning was logged, and
the invocation went out anyway. Without it written down the bucket looks
like a broken vendor name rather than a pointer back to the task log.
_resolve_agent_ref() wraps the identity lookup in a bare except Exception
so a subclass's broken agent_ref only costs a logged warning instead of
failing a call that already succeeded. That catch-all reached too far:
pydantic-ai's ModelRetry is control flow the calling model raises on
purpose, not a resolution failure, and this module already re-raises it
everywhere else agent_ref or invoke can produce one. Left unguarded, a
model's own retry request would have been silently discarded as yet
another "identity unresolved" warning.
Comment thread providers/common/ai/src/airflow/providers/common/ai/toolsets/managed_agent.py Outdated
…to a warning

_resolve_agent_ref() runs after invoke() has already produced an answer, and no
agent_ref implementation raises ModelRetry (only invoke_sync does, when the
client rejects a request). The except ModelRetry branch could therefore never be
reached, and if it were, re-raising would discard a good answer. Remove it so
every identity lookup failure is logged and resolved to None.
@Lee-W
Lee-W force-pushed the common-ai-managed-agent-review-followup branch from 3f2d2f2 to 3770cf4 Compare October 8, 2026 09:00
@Lee-W
Lee-W merged commit a0e9d68 into apache:main Oct 8, 2026
71 checks passed
@Lee-W
Lee-W deleted the common-ai-managed-agent-review-followup branch October 8, 2026 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants