Repository navigation
Conversation
Extract the drain.Helper setup into a shared constructor so drain diagnostics can inspect the same pod set the drainer uses. Also make the drain stall threshold configurable on the reconciler so tests can use a short threshold; the default remains five minutes. Assisted-by: AI Signed-off-by: Alice Frosi <afrosi@redhat.com>
When a drain stalls past the stall threshold, inspect the pods the drain would evict and report the PodDisruptionBudgets that prevent eviction. Emit a DrainBlockedByPDB event, re-emitting only when the set of blocking budgets changes, and surface blocked nodes in the pool status message. When every occupied reboot slot is held by a blocked drain and the rollout has consumed all of its disruption budget, report UpToDate=False with reason PDBBlocked. Partial blocks keep the RolloutInProgress reason so existing behavior is unchanged. Stalled drains are rechecked every 30 seconds, and the controller now has permission to get and list PodDisruptionBudgets. Assisted-by: AI Signed-off-by: Alice Frosi <afrosi@redhat.com>
Add an operations section covering the DrainBlockedByPDB event, the PDBBlocked UpToDate reason, and how to resolve each hint. Add the PDBBlocked reason to the architecture condition table. Assisted-by: AI Signed-off-by: Alice Frosi <afrosi@redhat.com>
Run a single-replica application with a zero disruption budget on a one-node pool and start a rollout. Verify the controller reports the blocking budget through a DrainBlockedByPDB event and the PDBBlocked pool condition while the node stays cordoned and does not reboot. Then relax the budget and verify the drain and rollout complete. Assisted-by: AI Signed-off-by: Alice Frosi <afrosi@redhat.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A node drain can stall forever when a PodDisruptionBudget (PDB) allows no evictions. Today the user sees only a generic
DrainTakingTooLongevent, with no hint about the cause.When a drain passes the stall threshold (5 minutes by default), the controller now:
DrainBlockedByPDBwarning on theBootcNodenaming each PDB and why it blocks (N pod(s) not ready,never allows eviction, or multiple PDBs selecting one pod). The event is re-emitted only when the blocking set changes.UpToDate=Falsewith reasonPDBBlockedwhen every occupied reboot slot is held by a blocked drain. If only some slots are blocked, the reason staysRolloutInProgressand the message names the node and PDB. Priority isPaused>PDBBlocked>RolloutInProgress.Rollout behavior is unchanged. This only adds reporting.
The pod and PDB eligibility checks follow the eviction API: pending, terminal and terminating pods are skipped, the unhealthy-pod eviction policy is honored, and a pod matched by 2 or more PDBs is reported as a misconfiguration.