backout down parent retry limiting in parent selection and nexthop strategies - #8546
Merged
Merged
Conversation
Contributor
|
@ywkaras volunteered to take a look at this. |
ywkaras
approved these changes
Dec 7, 2021
moonchen
pushed a commit
to moonchen/trafficserver
that referenced
this pull request
Mar 17, 2022
* asf/9.2.x: Updated ChangeLog docs: fix fedora install notes and spelling issues (apache#8537) Docs: Fix default value of proxy.config.ssl.handshake_timeout_in (apache#8574) Partial of revert "Cleanup generated LDFLAGS for jemalloc (apache#8285)" (apache#8533) TSUserArg: add value type checking (apache#8550) Relax key validation of sni.yaml (apache#8549) Clear random header value by AIO read error (apache#8559) Fixes macOS arm64 builds (again) (apache#8556) Traffic Dump: Use the correct transaction user index (apache#8548) combo_handler: Initialize User Arg Index in TSRemapInit (apache#8551) backout down parent retry limiting in parent selection and nexthop (apache#8546)
masaori335
added a commit
that referenced
this pull request
Aug 5, 2026
A parent that accepts connections but never sends a response was marked down and then restored to the pool by the very next retry probe, even when that probe also failed, so it kept receiving traffic indefinitely while healthy peers took none of the load. handle_response_from_parent() called markParentUp() on entry, before the switch on current.state, so a parent selected as a retry candidate was restored before ATS knew the retry's outcome. markParentUp() zeroes failedAt and failCount; the markParentDown() that follows a failed probe then takes its result->retry branch, which leaves new_fail_count at 0, so available is never re-cleared. The parent therefore returned to the pool with a cleared failure count after every retry_time window, no matter how long it stayed degraded. The CONNECTION_ALIVE branch already calls markParentUp() for a retry that actually succeeded, which is the correct restore point. Removing the entry call restores the behavior that predates #8546: that commit backed out the retry-limiting work and replaced a retryComplete() call -- which only decremented an in-flight retry counter and never touched availability -- with markParentUp(), silently turning "the retry finished" into "the parent is healthy". Two autests pin both outcomes of the retry path. A parent that is still silent when probed stays down; a parent that has recovered is restored. Each was negative-controlled against its own call site: reverting this change fails the first, and removing the CONNECTION_ALIVE call fails only the second.
cmcfarlen
pushed a commit
that referenced
this pull request
Aug 6, 2026
A parent that accepts connections but never sends a response was marked down and then restored to the pool by the very next retry probe, even when that probe also failed, so it kept receiving traffic indefinitely while healthy peers took none of the load. handle_response_from_parent() called markParentUp() on entry, before the switch on current.state, so a parent selected as a retry candidate was restored before ATS knew the retry's outcome. markParentUp() zeroes failedAt and failCount; the markParentDown() that follows a failed probe then takes its result->retry branch, which leaves new_fail_count at 0, so available is never re-cleared. The parent therefore returned to the pool with a cleared failure count after every retry_time window, no matter how long it stayed degraded. The CONNECTION_ALIVE branch already calls markParentUp() for a retry that actually succeeded, which is the correct restore point. Removing the entry call restores the behavior that predates #8546: that commit backed out the retry-limiting work and replaced a retryComplete() call -- which only decremented an in-flight retry counter and never touched availability -- with markParentUp(), silently turning "the retry finished" into "the parent is healthy". Two autests pin both outcomes of the retry path. A parent that is still silent when probed stays down; a parent that has recovered is restored. Each was negative-controlled against its own call site: reverting this change fails the first, and removing the CONNECTION_ALIVE call fails only the second. (cherry picked from commit 698cb9f)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
backs out the parent retry limiting in parent selection and next hop strategies due to issues found with this in production use.