[pankajkoti]

When a reviewer edits the output of an LLMOperator approval gate, the edited text is validated back into the operator's output_type. If validation failed, rehydrate_pydantic_output silently returned the raw string. The task succeeded and downstream tasks received a str instead of an int, a list or a model. For example output_type=int with the edit "thirty-one" produced the string 'thirty-one'.

It now raises ReviewedOutputValidationError, a ValueError subclass in the provider's exceptions module. The message names the output type, the text received (truncated) and the pydantic validation message, so the task fails with a readable error. Details:

• output_type=str still accepts any text.
• Types whose value is plain text (a str Literal, a str Enum, str | None) still accept bare text. For these the message reports the plain text failure instead of "Invalid JSON". A JSON edit with a typo, such as a trailing comma, keeps the JSON error, because the plain text attempt would only report a top-level type mismatch.
• Only an edit can fail the task. An unedited approval whose output cannot be validated back (for example a model with an irreversible serializer) returns the text unchanged and logs a warning, as on main.
• The HITL form hands back an object output as a parsed dict, which rounds integers above 2**53, writes 1.0 as 1 and may reorder keys. An untouched object is compared by value the way the browser parses it, so it is never read as an edit.
• An output_type that pydantic has no schema for, a pydantic-ai output function or marker (such as ToolOutput), and an output function whose parameters pydantic wraps in a definitions schema, still pass the text through. There is nothing to validate against, or validating would call the function again with the reviewed text. No warning is logged for this passthrough. A union with a str member, such as Model | str, keeps an edit that is not the model as text.

The approval gate (LLMOperator and its subclasses) builds the review string with dump_output_to_json. It dumps by field name with round_trip=True, and validation accepts a field name, and an alias unless the model sets validate_by_alias=False. So models with aliases, including serialization-only aliases, validate back, and computed fields are left out. The decision-policy review body now shows the same JSON as the edit field. AgentOperator never validates its review text back, so it uses dump_output_for_display, a plain JSON dump that keeps computed fields visible to the reviewer. Both fall back to str() with a warning for values pydantic cannot serialize.

LLMSchemaCompareOperator re-validated the string after the parent returned and raised its own ValueError. That fallback is no longer reachable, so the override is removed. The new exception is still a ValueError.

Behavior change: an invalid reviewer edit now fails the task instead of passing a string downstream.

Related PRs: #74537 (merged) made AgentOperator return the output it already holds instead of converting the approved text, so this PR no longer touches its review loop apart from regenerated output using the shared helper. #70132 (open) also edits rehydrate_pydantic_output and mixins/hitl_review.py, and #74493 (open) overlaps operators/llm.py. Whichever merges later will need a rebase. A note for the behavior change is added to the provider changelog.

Follow-up idea, not part of this PR: validate the edit when the reviewer submits the form, so they see the error immediately instead of the task failing after approval.

---

Was generative AI tooling used to co-author this PR?

• Yes (Claude Code, Sonnet 5.5)

Generated-by: Claude Code (Sonnet 5.5) following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

🤖 Generated with Claude Code

[kaxil]

:( test failure:

FAILED providers/common/ai/tests/unit/common/ai/utils/test_output_type.py::TestRehydratePydanticOutput::test_error_reports_the_json_error_for_plain_text_sent_to_a_structured_type[arguments_type] - AssertionError: Regex pattern did not match.
  Expected regex: 'Invalid JSON'
  Actual message: "The reviewed output could not be converted to the output_type Point. Received 'hello'. 1 validation error for Point\n  Input should be a tuple, list, dictionary or an instance of Point [type=named_tuple_type, input_value='hello', input_type=str]\n    For further information visit https://errors.pydantic.dev/2.14/v/named_tuple_type"

https://github.com/apache/airflow/actions/runs/38529714144/job/115743582407?pr=74473

[pankajkoti]

:( test failure:

FAILED providers/common/ai/tests/unit/common/ai/utils/test_output_type.py::TestRehydratePydanticOutput::test_error_reports_the_json_error_for_plain_text_sent_to_a_structured_type[arguments_type] - AssertionError: Regex pattern did not match.

Yes :) Looks like the test was newly added in the recent commit a4f58a9. It seems to have failed because pydantic 2.14 renamed the NamedTuple error from arguments_type to named_tuple_type, which the set of container mismatch errors did not list. I attempted a fix in 2139a29 by adding named_tuple_type to the list

[kaxil]

:( test failure:

FAILED providers/common/ai/tests/unit/common/ai/utils/test_output_type.py::TestRehydratePydanticOutput::test_error_reports_the_json_error_for_plain_text_sent_to_a_structured_type[arguments_type] - AssertionError: Regex pattern did not match.

Yes :) Looks like the test was newly added in the recent commit a4f58a9. It seems to have failed because pydantic 2.14 renamed the NamedTuple error from arguments_type to named_tuple_type, which the set of container mismatch errors did not list. I attempted a fix in 2139a29 by adding named_tuple_type to the list

Yeah, I was hoping to get CI to green at midnight to merge to this in 1.0.0rc1 😬