Repository navigation
Status of testing Apache Airflow 3.0.4rc2 and TaskSDK 1.0.4rc1 #54124
Description
We are kindly requesting that contributors to Apache Airflow RC 3.0.4rc1 help test the RC.
Please let us know by commenting if the issue is addressed in the latest RC.
- use of debugpy to run airflow components via breeze (#51763): @rawwar
- [Backport v3-0-test] Add back dag parsing pre-import optimization (#50371) (#52698): @Lzzz666 @kaxil
Linked issues: - [v3-0-test] Dev: Fix KeyError in release issue content generation (#52751) (#52756): @kaxil
- [v3-0-test] Advanced auto classification for provider documentation (#52902) (#52925): @amoghrajesh
- [v3-0-test] Workaround the need to use latest version on Lucas pre-commit's (#52931) (#52942): @potiuk
- [v3-0-test] Make the update_installer_and_pre_commit script more resilient to re matches (#52960) (#52961): @amoghrajesh
- [v3-0-test] Remove upper-binding for "python-requires" (#52980) (#52984): @potiuk
- [v3-0-test] Remove remnants of ~= used in requires-python. (#52985) (#52987): @potiuk
Linked issues: - [v3-0-test] Unify connection not found exceptions between AF2 and AF3 (#52968) (#53093): @Lzzz666 @github-actions[bot] @potiuk @amoghrajesh
- [v3-0-test] docs: update public interface doc to reflect airflow.sdk and AIP-72 (#52197) (#53117): @nrnavaneet
- Run Task failure callbacks on DAG Processor when task is externally killed (#53058) (#53143): @kaxil
Linked issues: - [v3-0-test] Resolve OOM When Reading Large Logs in Webserver (#53167): @jason810496
Linked issues: - [v3-0-test] Move --no-binary flag to uv command line from pyproject.toml (#53179) (#53185): @potiuk
- [v3-0-test] Install python from source again (#53150) (#53202): @aritra24 @potiuk
- [v3-0-test] Fix package typo in build images (#53209) (#53210): @gopidesupavan
- [v3-0-test] Bump uv to 0.7.20 (#53207) (#53220): @potiuk @gopidesupavan
- [v3-0-test] Pass GITHUB_TOKEN to update checks (#53219) (#53221): @potiuk
- [v3-0-test] Bring back quick image build check and increase timeout minutes (#53227) (#53230): @potiuk @gopidesupavan
Linked issues: - [v3-0-test] bump remaining uv versions (#53228) (#53232): @bugraoz93 @potiuk
- [v3-0-test] fix(ci): bump python versions (#53229) (#53233): @bugraoz93
- [v3-0-test] Update mypy script with --warn-unused-ignores (#53240) (#53245): @gopidesupavan
- [v3-0-test] Fix mypy cmd (#53246) (#53247): @gopidesupavan
- [v3-0-test] Fix broken link in contributors doc on providers (#53344) (#53347): @zach-overflow
- [v3-0-test] fix: Make
task_success_overtimeconfigurable (#53342) (#53351): @kacpermuda - [v3-0-test] Remove gunicorn daemonize for api-server (#52860) (#53372): @jason810496
Linked issues: - [v3-0-test] Update docs for Airflow CLI help command (#51042) (#53383): @rapsealk
- [v3-0-test] Update doc string for BaseOperator (#53403) (#53404): @eladkal
- [v3-0-test] Fixing uv upgrade failures on main (#53406) (#53451): @potiuk @amoghrajesh
- [v3-0-test] Make GITHUB_TOKEN variable mandatory when updating installers (#53452) (#53457): @potiuk
- [v3-0-test] DagBag in file parsing used to collect dags in dag processing uses default value for safe_mode, which resolves to value in configuration (#52694) (#53507): @BBQing
- [v3-0-test] Handle invalid execution API urls gracefully in supervisor (#53082) (#53518): @amoghrajesh
- [v3-0-test] Remove warning for
BaseOperator.executorbecause its false (#53496) (#53519): @amoghrajesh - [v3-0-test] Add changelog for regenerated image for Airflow 3.0.3 (#5… (#53522): @potiuk
- [v3-0-test] Add sql extras to pandas in the providers where sqlalchem… (#53539): @potiuk
- [v3-0-test] Upgdate to uv 0.8.0 (#53552) (#53564): @potiuk
- [v3-0-test] Include fixed navbar for anchor position calculation. (#5… (#53581): @simi @potiuk
- [v3-0-test] fix: accordian child contents to be visible when overflown (#53595) (#53602): @naseemshah
- [v3-0-test] fix(systemd): Update systemd unit files and documentation for Airflow 3.0 (#52294) (#53609): @nrnavaneet
- [v3-0-test] Skip empty dag run conf rows and set statement timeout (#50788) (#53619): @dstandish
- [v3-0-test] Add deprecation notice for using Connection from models (#53594) (#53621): @amoghrajesh
- [v3-0-test] Ignore cursor rules from the git tree (#53658) (#53659): @amoghrajesh
- Fix incorrect reference to SimpleHttpOperator in 3.0.0 release notes (#53661): @eladkal
- [v3-0-test] Fix a UI bug where "All" does not show all dags when hide_paused_dags_by_default = True (#53656) (#53672): @zachliu
- [v3-0-test] Add GITHUB_TOKEN in test helm release job (#53674) (#53677): @potiuk
- [v3-0-test] Add possibility to apply a commit with patch to docs when publishing (apache#53668) (#53678): @potiuk
- [v3-0-test] Improve validation of breeze workflow-run command (#53685) (#53688): @potiuk
- [v3-0-test] Fix typo in AIRFLOW_COMMIT(S) diagnostics in workflow_run (#53686) (#53689): @potiuk
- [v3-0-test] CHG: cover v1, v2 (#53636) (#53709): @kyungjunleeme
- [v3-0-test] Bump gitpython from 3.1.44 to 3.1.45 in /airflow-core (#5… (#53731): @dependabot[bot] @potiuk
- [v3-0-test] Bump gitpython from 3.1.44 to 3.1.45 (#53724) (#53732): @dependabot[bot] @potiuk
- [v3-0-test] Unconditionally disable start_from_trigger (#53744) (#53750): @uranusjr
- [v3-0-test] Allow Remote logging providers to load connections from the API Server (#53719) (#53761): @ashb
- [v3-0-test] Remove extra slash from endpoint URL (#53755) (#53764): @lpillmann
- [v3-0-test] Respect
appsflags for api_server_command (#52929) (#53775): @jason810496
Linked issues: - [v3-0-test] Make branch tests more resilient to provider dependency c… (#53777): @potiuk
- [v3-0-test] Better type handling to account for types-docutils bump (… (#53778): @potiuk @amoghrajesh
- [v3-0-test] sphinx: avoid repeated isinstance checks (#53367) (#53783): @Dev-iL @gopidesupavan
- [v3-0-test] Add documentation for self-signed certificate (#53788) (#53792): @danieldean
- [v3-0-test] Add cert used for apiserver to client (fix for self-signe… (#53793): @danieldean @potiuk
- [v3-0-test] Add user facing docs for running a separate Task Execution API-server (#53789) (#53794): @jason810496
- [v3-0-test] Remove unnecessary group_by clause in event logs query (#53733) (#53807): @xiaohuanlin @gvnn-gov
Linked issues: - [v3-0-test] Make BaseOperator on_kill functionality work with TaskSDK (#53832): @gopidesupavan
- [v3-0-test] Serialize nan and infinities to string (#53835) (#53844): @uranusjr
- [v3-0-test] Fix scheduler heartbeat timeout failures with
DetachedInstanceError(#53838) (#53858): @kaxil - [v3-0-test] Revert "Fix editing connection with sensitive extra field (#52403)" (#53889): @pierrejeambrun
- docs: Update DAG author as Dag author (#53857) (#53950): @Lee-W
- [v3-0-test] Removed bold formatting for Public Interface for Airflow 3.0+ (#53955) (#53964): @sunank200
- Handle unmodified sensitived fields when updating connections (#53943) (#53973): @pierrejeambrun
- [v3-0-test] Add type annotation for dev system tests (#53979) (#53996): @amoghrajesh
- [v3-0-test] Fix nth-child warning (#53982) (#54000): @pierrejeambrun
- Revert "[v3-0-test] Remove gunicorn daemonize for api-server (#52860) (#53372)" (#54002): @jason810496 @jedcunningham
Linked issues: - [v3-0-test] Fix version for
parsing_pre_import_modulesconfig option move (#53995) (#54011): @jedcunningham - [v3-0-test] Fix AIRFLOW_API_APPS constant in api_server_command (#54007) (#54012): @jason810496
- [v3-0-test] fix(diagrams): update labels from "Webserver(s)" to "API Server(s)" i… (#54020): @potiuk @aayushbisen
- [v3-0-test] build: update uv to 0.8.4 and pip to 25.2 (#53954) (#54021): @Lee-W @potiuk
- [v3-0-test] Unmount dialog & add onOpen (#53307) (#54025): @pierrejeambrun @Jasperora
- [v3-0-test] Flexible form use ReactMarkdown instead of default Markdo… (#54040): @pierrejeambrun @jscheffl
- [v3-0-test] Fix
_extract_template_fieldsmypy error (#53855) (#54047): @choo121600 - [v3-0-test] Make log redaction safer in edge case when redaction has an error (#54046) (#54048): @ashb
- [v3-0-test] Only forward debugger ports when debug components are used (#54051) (#54053): @rawwar @potiuk
Linked issues: - [v3-0-test] Simplify installing airflow from GitHub repo in breeze (#54070) (#54076): @potiuk
- Limit paramiko to
< 4.0.0till we remove DSS support (#54078): @amoghrajesh
Thanks to all who contributed to the release (probably not a complete list!):
@Lzzz666 @jscheffl @gvnn-gov @aayushbisen @zach-overflow @bugraoz93 @jason810496 @sunank200 @jedcunningham @lpillmann @zachliu @dstandish @potiuk @kyungjunleeme @amoghrajesh @xiaohuanlin @Lee-W @gopidesupavan @Jasperora @nrnavaneet @BBQing @aritra24 @rawwar @rapsealk @dependabot[bot] @kacpermuda @uranusjr @naseemshah @ashb @pierrejeambrun @danieldean @simi @choo121600 @kaxil @eladkal @Dev-iL
Activity
potiuk commentedon Aug 5, 2025
Tested and checked all my changes - they look good. I found one issue with #54020 -> there was another cherry-pick missing - #54129 -> which caused generated diagrams to miss icons (see here: https://airflow.staged.apache.org/docs/apache-airflow/stable/core-concepts/overview.html) . I am cherry-picking the missing change to v3-0-test (#54129). No need to fix anything - we will just have to add the commit (alongside #54100) when building documentation.
amoghrajesh commentedon Aug 5, 2025
CI / Dev tool fixes:
< 4.0.0till we remove DSS support #54078Actual fixes:
BaseOperator.executorbecause its false (#53496) #53519All of it works as expected.
pierrejeambrun commentedon Aug 5, 2025
Tested a bunch of my changes
bugraoz93 commentedon Aug 5, 2025
My changes look good. Thanks, Ash!
danieldean commentedon Aug 5, 2025
#53793 works perfectly. Thanks!
BBQing commentedon Aug 5, 2025
#53507 Tested and it works
naseemshah commentedon Aug 5, 2025
My change #53602 works well!
Lee-W commentedon Aug 6, 2025
Tested a few dags with airflow==3.0.4rc1 and task-sdk==1.0.4rc1. Works fine.
One issue I encountered was that after running
uv add apache-airflow==3.0.4rc1 --prerelease=allow, it does not use task-sdk==1.0.4rc1. Additionally, airflow 3.0.4rc1 does not work with task-sdk 1.0.3. Not sure whether we need to pin any depschoo121600 commentedon Aug 6, 2025
My change only updates pre-commit scripts, so no issues with this rc 😉
aritra24 commentedon Aug 6, 2025
My changes were with the build system so shouldn't effect the rc
Lzzz666 commentedon Aug 6, 2025
Tested with 3.0.4rc1:
Looks good from my side.
[-]Status of testing Apache Airflow 3.0.4rc1 and TaskSDK 1.0.4rc1[/-][+]Status of testing Apache Airflow 3.0.4rc2 and TaskSDK 1.0.4rc1[/+]ashb commentedon Aug 6, 2025
Fixed in apache-airflow rc2
gopidesupavan commentedon Aug 6, 2025
Verified my changes LGTM, ran few examples working fine.
I find one issue that is related to standard provider example dag while running
PythonVirtualenvOperator. getting below errorThis was introduced by this https://github.com/apache/airflow/pull/53390/files#diff-80d85a10fe6b4cea5c06a9959a893c000cc90fd04ff1b0c0344ef937b0251111R892
Going to fix it.
gopidesupavan commentedon Aug 7, 2025
raised here #54214
Lee-W commentedon Aug 8, 2025
Tested with the latest fix with some Dags. The resolution works fine and the Dags work fine
amoghrajesh commentedon Aug 8, 2025
Tested the latest compatibility fix and my other changes as reported in #54124 (comment).
It rightly installs 1.0.4rc1 of task sdk too.
If i force it with older sdk: