fix: warn about hardcoded 24h visibility_timeout that kills long-running Celery tasks

[YoannAbriel]

fix: warn about hardcoded 24h visibility_timeout that kills long-running Celery tasks

Problem

When using the Celery executor with Redis/SQS brokers and no explicit visibility_timeout configured, Airflow silently applies a default of 86400 seconds (24 hours) in default_celery.py. Tasks running longer than 24 hours are terminated by the broker redelivering the message to another worker, which fails with ServerResponseError('Invalid auth token: Signature has expired'). Users have no indication this limit exists or how to change it.

The task_acks_late configuration description also incorrectly states it "effectively overrides the visibility timeout", which is not true for Redis/SQS brokers — the broker-level redelivery happens regardless of acknowledgment settings.

Root Cause

get_default_celery_config() in default_celery.py hardcodes broker_transport_options["visibility_timeout"] = 86400 when no value is configured and the broker is Redis or SQS. This happens silently with no log output, so users don't know about the 24h limit until their tasks are killed. The task_acks_late documentation in get_provider_info.py compounds the confusion by claiming it overrides visibility_timeout.

Fix

Three changes in the celery provider:

1. default_celery.py: Added a log.warning() when the default 86400s visibility_timeout is applied, telling users about the limit and how to increase it via [celery_broker_transport_options] visibility_timeout.

2. get_provider_info.py (task_acks_late description): Corrected the documentation to state that task_acks_late does NOT override visibility_timeout for Redis/SQS brokers, and that users must separately increase visibility_timeout for long-running tasks.

3. get_provider_info.py (visibility_timeout description): Added note that Airflow defaults to 86400s when not configured, and that tasks exceeding this value will be terminated.

Added 3 unit tests:

• test_visibility_timeout_default_warns_when_not_configured — verifies warning is logged with Redis broker
• test_visibility_timeout_no_warning_when_configured — verifies no warning when explicitly set
• test_visibility_timeout_not_set_for_unsupported_broker — verifies no warning/default for RabbitMQ

Tests follow existing patterns in test_celery_executor.py (validated via CI).

Closes: #62218

---

Was generative AI tooling used to co-author this PR?

• Yes — Claude Code

Generated-by: Claude Code following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.