Stabilize post and slow_post autests under parallel runs - #12886
Conversation
Gate each check_threads invocation on ATS port readiness using When.PortOpen(ts.Variables.port) to avoid races where validation runs before traffic_server is ready.
Add explicit origin readiness checks in post-early-return and accept the current microserver exception variant in server_abort so these tests pass consistently on Fedora 43.
There was a problem hiding this comment.
Pull request overview
This PR improves determinism in the thread_config gold test by ensuring the thread-validation helper (check_threads.py) does not run until the corresponding ATS instance is actually accepting connections on its configured port, eliminating a startup-order race in parallel autest runs.
Changes:
- Add per-run readiness gating so each
check_threads.pyinvocation waits onWhen.PortOpen(ts.Variables.port)before proceeding. - Apply the same readiness condition consistently across all thread-configuration variants in the test.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Revert the thread_config PortOpen readiness change and keep ATS process startup gating on its existing fully-initialized readiness behavior. For post-early-return, move upstream readiness to server process Ready fields instead of passing ready via StartBefore calls.
|
[approve ci autest 1] |
The nc-l based server1.sh only accepts a single TCP connection, which
is consumed by the When.PortOpen() readiness probe, leaving nothing for
ATS to connect to. Replace with a new shared mock_origin.py tool that:
- Absorbs PortOpen readiness probes (connect/disconnect with no data)
- Sends the configured HTTP response after reading request headers
- Drains remaining request data before closing, preventing TCP RST
that causes ATS to return 502 on HTTP/2 streams
Tested 5/5 parallel ASAN runs passing on hera2.
|
The consistent AuTest 1of4 failure was caused by a fundamental incompatibility between
The fix (pushed in
Tested 5/5 parallel ASAN runs passing on a local dev machine. The previous |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
I see this error on my PR as well. |
|
Can we remove server1.sh ? |
|
Yes, it can be removed. I will push a new commit to delete it. |
* Stabilize thread_config autest startup ordering -- gate check_threads on ATS port readiness to avoid validation races. * Stabilize post/slow_post autests -- add explicit origin readiness checks in post-early-return; accept microserver exception variant for Fedora 43. * Revert thread_config PortOpen change per review -- keep ATS startup gating on existing readiness behavior, move upstream readiness to server Ready fields. * Replace nc-based server with mock_origin.py -- server1.sh only accepts one TCP connection (consumed by readiness probe). mock_origin.py absorbs probes, sends configured response, and drains request data to prevent TCP RST / 502. * Remove unused server1.sh replaced by mock_origin.py (cherry picked from commit a1ac1db)
…3525) * Stabilize thread_config autest startup ordering -- gate check_threads on ATS port readiness to avoid validation races. * Stabilize post/slow_post autests -- add explicit origin readiness checks in post-early-return; accept microserver exception variant for Fedora 43. * Revert thread_config PortOpen change per review -- keep ATS startup gating on existing readiness behavior, move upstream readiness to server Ready fields. * Replace nc-based server with mock_origin.py -- server1.sh only accepts one TCP connection (consumed by readiness probe). mock_origin.py absorbs probes, sends configured response, and drains request data to prevent TCP RST / 502. * Remove unused server1.sh replaced by mock_origin.py (cherry picked from commit a1ac1db) Co-authored-by: Bryan Call <bcall@apache.org>
Problem
The
post-early-returnandslow_post/server_abortautests fail intermittently under parallel execution on Fedora 43. Thenc -lbased mock server only accepts a single TCP connection, which gets consumed by the readiness probe, leaving nothing for ATS to connect to.Changes
thread_configstartup ordering -- gate eachcheck_threadsinvocation on ATS port readiness usingWhen.PortOpen(ts.Variables.port)to avoid races where validation runs beforetraffic_serveris ready.postandslow_postautests -- add explicit origin readiness checks inpost-early-returnand accept the current microserver exception variant inserver_abortfor Fedora 43 compatibility.thread_configPortOpen change per review -- keep ATS startup gating on existing fully-initialized readiness behavior. Movepost-early-returnupstream readiness to server processReadyfields instead ofStartBeforecalls.nc-based server withmock_origin.py--server1.shonly accepts a single TCP connection, consumed by theWhen.PortOpen()readiness probe.mock_origin.pyabsorbs probes, sends the configured HTTP response, and drains request data before closing to prevent TCP RST that causes 502 on H2 streams.server1.sh-- replaced bymock_origin.py.Testing