You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
refresh-who-has can break the worker state machine #6528
Two tasks, x and y, are only available on a busy worker.
The worker sends request-refresh-who-has to the scheduler.
The scheduler responds with refresh-who-has, stating that x has become missing, while y has gained an additional replica.
The handler for RefreshWhoHasEvent empties x.who_has and recommends a transition to missing.
Before the recommendation can be implemented, the same event handler invokes _ensure_communicating to allow y to transition to flight. This in turn pops x from data_needed - but x has an empty who_has, which trips an AssertionError in _ensure_communicating.
This in turn trips @fail_hard, which violently shuts down the whole worker.
Activity
crusaderky commentedon Jun 8, 2022
FYI @jrbourbeau @fjetter