Repository navigation
Detect stalled STT connections and limit recovery retries - #8123
Merged
Merged
Conversation
For aiohttp-based streaming STT, track all incoming WebSocket messages. Send a ping after 2 seconds without a message. Report a connection failure after 6 seconds without a message. The timeout is customizable with `liveness_timeout`. Reset the retry count when the provider reports processing progress. Limit retries when no progress occurs, so the fallback adapter can select another provider. Fix retry handling in AWS, Google, and NVIDIA STT.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
theomonnom
approved these changes
Oct 26, 2026
pradneshfernandez
added a commit
to pradneshfernandez/agents
that referenced
this pull request
Oct 26, 2026
Transcribe timeouts now go through the base retry (livekit#8123), so the test needs a retry budget to reach the second attempt.
This was referenced Oct 26, 2026
Rehansanjay
added a commit
to Rehansanjay/agents
that referenced
this pull request
Oct 27, 2026
baseten, murf, neuphonic, resemble, telnyx and xai have no liveness check on their TTS websocket. If the socket goes half-open after connecting -- the provider, or a NAT in between, gone without a FIN or an RST -- the text is pushed, no audio comes back, nothing raises, and the agent is silently mute for the rest of the call. `tts.FallbackAdapter` is never told. livekit#8123 added `utils.aio.WebSocketMonitor` and a `liveness_timeout` connect option, and every streaming STT plugin now reads through it. That helper is not STT-specific and the TTS side has not adopted it, so this does the same thing here: connect with `autoping=False`, build one monitor per connection, and read through `monitor.receive()`. baseten and telnyx read with `async for msg in ws`, which cannot carry a monitor, so both become explicit loops; their existing close and error branches are unchanged. Measured against a local server that accepts the websocket upgrade and then sends nothing and answers no pings, with the default `liveness_timeout` of 6.0: | plugin | before | after | | --------- | ---------- | ----- | | baseten | never, 75s | 7.6s | | murf | never, 75s | 6.2s | | neuphonic | never, 75s | 6.1s | | resemble | never, 75s | 6.1s | | telnyx | never, 75s | 6.1s | | xai | never, 75s | 6.1s | And against a peer that answers pings but returns no audio -- a provider that is simply slow -- all six stay connected past 20s, because the pongs keep resetting the timer. That case is why this reads through the monitor rather than putting a plain deadline on the receive: a passive deadline cannot tell a slow provider from a dead one, and would cut off a healthy stream waiting on a slow first token. Probe and raw numbers: https://github.com/Rehansanjay/livekit-tts-halfopen
This was referenced Oct 27, 2026
himanshumehta
added a commit
to himanshumehta/agents
that referenced
this pull request
Nov 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For aiohttp-based streaming STT, track all incoming WebSocket messages. Send a ping after 2 seconds without a message. Report a connection failure after 6 seconds without a message.
The timeout is customizable with
liveness_timeout.Reset the retry count when the provider reports processing progress. Limit retries when no progress occurs, so the fallback adapter can select another provider.
Fix retry handling in AWS, Google, and NVIDIA STT.