Skip to content

fix(providers/standard): improve error message for non-serializable op_kwargs in virtualenv operators - #63270

Merged
potiuk merged 2 commits into
apache:mainfrom
YoannAbriel:fix/issue-61741
Mar 12, 2026
Merged

fix(providers/standard): improve error message for non-serializable op_kwargs in virtualenv operators#63270
potiuk merged 2 commits into
apache:mainfrom
YoannAbriel:fix/issue-61741

Conversation

@YoannAbriel

Copy link
Copy Markdown
Contributor

Problem

When render_template_as_native_obj=True on a DAG, templates like {{ ti }} in op_kwargs resolve to live TaskInstance objects instead of strings. When PythonVirtualenvOperator._write_args tries to pickle these for the subprocess, it fails with a PicklingError from structlog internals (Only BytesLoggers to sys.stdout and sys.stderr can be pickled). The error gives no indication that the root cause is a non-serializable kwarg value.

Root Cause

_write_args calls self.pickling_library.dumps(...) on the full args dict without any pre-validation. When a value can't be pickled, the raw exception propagates — and since TaskInstance contains structlog loggers, the error message references structlog internals rather than the actual problem.

Fix

Wrap the dumps call in a try/except. On failure, iterate over op_kwargs to identify which specific keys hold non-serializable values, then raise an AirflowException naming the bad keys and suggesting alternatives (e.g. {{ ti.task_id }} instead of {{ ti }}).

Added a unit test that verifies the improved error message when a non-serializable object is passed in op_kwargs.

Closes: #61741


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code

Generated-by: Claude Code following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@potiuk

potiuk commented Mar 10, 2026

Copy link
Copy Markdown
Member

@YoannAbriel This PR has been converted to draft because it does not yet meet our Pull Request quality criteria.

Issues found:

  • Pre-commit / static checks: Failing: CI image checks / Static checks. Run prek run --from-ref main locally to find and fix issues. See Pre-commit / static checks docs.
  • mypy (type checking): Failing: CI image checks / MyPy checks (mypy-airflow-core). Run prek --stage manual mypy-airflow-core --all-files locally to reproduce. You need breeze ci-image build --python 3.10 for Docker-based mypy. See mypy (type checking) docs.
  • Provider tests: Failing: Postgres tests: providers / DB-prov:Postgres:14:3.10:-amazon,celer...standard, MySQL tests: providers / DB-prov:MySQL:8.0:3.10:-amazon,celer...standard, Sqlite tests: providers / DB-prov:Sqlite:3.10:-amazon,celer...standard, Non-DB tests: providers / Non-DB-prov::3.10:-amazon,celer...standard, Special tests / Min SQLAlchemy test: providers / DB-prov:MinSQLAlchemy-Postgres:14:3.10:-amazon,celer...standard (+6 more). Run provider tests with breeze run pytest <provider-test-path> -xvs. See Provider tests docs.
  • Other failing CI checks: Failing: CI image checks / Test Python API client, Postgres tests: core / DB-core:Postgres:14:3.10:API...Serialization, MySQL tests: core / DB-core:MySQL:8.0:3.10:API...Serialization, Sqlite tests: core / DB-core:Sqlite:3.10:API...Serialization, Non-DB tests: core / Non-DB-core::3.10:API...Serialization (+8 more). Run prek run --from-ref main locally to reproduce. See static checks docs.

Note: Your branch is 48 commits behind main. Some check failures may be caused by changes in the base branch rather than by your PR. Please rebase your branch and push again to get up-to-date CI results.

What to do next:

  • The comment informs you what you need to do.
  • Fix each issue, then mark the PR as "Ready for review" in the GitHub UI - but only after making sure that all the issues are fixed.
  • Maintainers will then proceed with a normal review.

Converting a PR to draft is not a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. If you have questions, feel free to ask on the Airflow Slack.


Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.

…p_kwargs in PythonVirtualenvOperator

When render_template_as_native_obj=True, templates like {{ ti }} resolve to
live Airflow objects that can't be pickled. The resulting PicklingError from
structlog internals is confusing and doesn't point to the root cause.

Catch serialization failures in _write_args, identify which specific kwargs
failed, and raise a clear AirflowException naming the bad keys and suggesting
to use string attributes instead.

Closes: apache#61741
@YoannAbriel
YoannAbriel marked this pull request as ready for review March 11, 2026 07:19
@potiuk
potiuk merged commit a93dc92 into apache:main Mar 12, 2026
132 checks passed
PascalEgn pushed a commit to PascalEgn/airflow that referenced this pull request Mar 12, 2026
…p_kwargs in PythonVirtualenvOperator (apache#63270)

When render_template_as_native_obj=True, templates like {{ ti }} resolve to
live Airflow objects that can't be pickled. The resulting PicklingError from
structlog internals is confusing and doesn't point to the root cause.

Catch serialization failures in _write_args, identify which specific kwargs
failed, and raise a clear AirflowException naming the bad keys and suggesting
to use string attributes instead.

Closes: apache#61741
@YoannAbriel
YoannAbriel deleted the fix/issue-61741 branch March 12, 2026 10:05
Pyasma pushed a commit to Pyasma/airflow that referenced this pull request Mar 13, 2026
…p_kwargs in PythonVirtualenvOperator (apache#63270)

When render_template_as_native_obj=True, templates like {{ ti }} resolve to
live Airflow objects that can't be pickled. The resulting PicklingError from
structlog internals is confusing and doesn't point to the root cause.

Catch serialization failures in _write_args, identify which specific kwargs
failed, and raise a clear AirflowException naming the bad keys and suggesting
to use string attributes instead.

Closes: apache#61741
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PythonVirtualenvOperator: native template rendering breaks {{ ti }} / {{ task_instance }} pickling

2 participants