Recover the revoked-token check from a stale DB connection (fixes idle-timeout 500) - #71444
Open
uplsh580 wants to merge 1 commit into
Open
Recover the revoked-token check from a stale DB connection (fixes idle-timeout 500)#71444uplsh580 wants to merge 1 commit into
uplsh580 wants to merge 1 commit into
Conversation
uplsh580
added a commit
to uplsh580/airflow
that referenced
this pull request
Aug 11, 2026
uplsh580
added a commit
to uplsh580/airflow
that referenced
this pull request
Aug 11, 2026
uplsh580
force-pushed
the
fix/revoked-token-stale-session-retry-71395
branch
from
August 11, 2026 16:06
9107fe5 to
6850a84
Compare
uplsh580
marked this pull request as ready for review
August 11, 2026 16:06
vincbeck
reviewed
Aug 11, 2026
uplsh580
force-pushed
the
fix/revoked-token-stale-session-retry-71395
branch
2 times, most recently
from
August 11, 2026 23:28
7781cc1 to
291c92a
Compare
vincbeck
reviewed
Aug 12, 2026
uplsh580
force-pushed
the
fix/revoked-token-stale-session-retry-71395
branch
from
August 12, 2026 13:23
603906a to
c486ed8
Compare
The JWT revocation check added in 3.2.0 (RevokedToken.is_revoked in BaseAuthManager.get_user_from_token) is the first DB access in the auth path and runs on the shared scoped session. A prior request (e.g. FAB's deserialize_user) can leave that session bound to a connection the database later drops on idle timeout (MySQL error 4031, "disconnected ... because of inactivity"). The connection is never re-checked-out, so pool_pre_ping / pool_recycle cannot detect it, and the first authenticated request after an idle period fails with HTTP 500. deserialize_user already recovers from this class of failure (apache#62919), but the revocation check runs earlier in get_user_from_token and had no recovery, so the 500 simply returned one step sooner. Apply the same discard-and-retry: on SQLAlchemyError, remove the poisoned scoped session and retry the check once on a fresh connection. Related to apache#71395
uplsh580
force-pushed
the
fix/revoked-token-stale-session-retry-71395
branch
from
August 12, 2026 13:25
c486ed8 to
f54e0ea
Compare
vincbeck
approved these changes
Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recover the JWT revoked-token check from a stale pooled DB connection so the first authenticated request after an idle period no longer returns HTTP 500.
Issue: #71395
Problem
With MySQL as the metadata DB and the FAB auth manager, the first authenticated request after the api-server sits idle longer than MySQL's
wait_timeoutfails with HTTP 500. The exception is anOperationalError(MySQL error 4031, "The client was disconnected by the server because of inactivity") raised fromRevokedToken.is_revokedinsideBaseAuthManager.get_user_from_token. The immediately following request succeeds, because the failed query resets the pooled connection — so in production it looks like intermittent 500s after overnight idle periods.Root cause
settings.Session. A prior request (e.g.FabAuthManager.deserialize_user, or FAB's Flask views) can leave that scoped session bound to a connection the database later drops on idle timeout. Because the connection is never returned to the pool, there is no checkout event —pool_pre_ping/pool_recyclecannot help.FabAuthManager.deserialize_user.RevokedToken.is_revokedinget_user_from_tokennow runs beforedeserialize_user, so it is the first thing to touch the poisoned scoped session — and it had no recovery logic. The 500 that fix(fab): recover from first idle MySQL disconnect in token auth #62919 fixed is back, just raised one step earlier in the auth path.Fix
Give the revoked-token check the same recovery
deserialize_useralready has: onSQLAlchemyError, discard the scoped session (settings.Session.remove()) and retry the check once on a fresh connection. The recovery is factored into a smallBaseAuthManager._is_token_revokedhelper soget_user_from_tokenstays readable.This is the per-call-site variant proposed in the issue (the one that has been running in production for several weeks and eliminated these 500s). It is minimal and mirrors the established #62919 pattern rather than introducing a new session-management abstraction. A persistent DB error still surfaces — only a single transient stale-connection error is absorbed.
Tests
Added two regression tests in
test_base_auth_manager.py. Both raise a 4031-styleOperationalErroron the firstis_revokedcall and assert the scoped session is discarded (settings.Session.remove()) and the check is retried exactly once:InvalidTokenError("Token has been revoked")is still raised.related: #71395
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code following the guidelines