Repository navigation
fix(voice): recover STT stream after unrecoverable errors - #6418
Merged
Merged
Conversation
Fixes #5665. Unlike the LLM and TTS streams, which are recreated every turn, the STT stream is long-lived: it is created once per agent activity and iterated by a single pump task. When it exhausted its per-utterance retries it emitted an unrecoverable stt_error and the pump ended, so nothing recreated it — and because stt_error had none of the session-level tolerance that llm_error/tts_error have, the session closed immediately on the first unrecoverable STT error. This makes the long-lived STT stream self-healing and gives it the same session-level tolerance: - _STTPipeline recreates the STT stream (with a short backoff) after an unrecoverable error, so the agent keeps hearing the user if the provider recovers. - AgentSession applies max_unrecoverable_errors to stt_error as the circuit breaker: each failed stream increments the count, and it resets on a user transcript (STT's success signal, mirroring how llm/tts reset on agent speaking). - The pump does not recreate the stream while the session is closing, so a teardown in progress can't spawn provider connections that are torn down at once. Supersedes #5666, #5997, and #6001, which added the session counter alone — without the reconnect the counter freezes at 1 (a dead stream emits no further errors) and the session stays open but deaf.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This was referenced Jul 14, 2026
Narrow the pump's reconnect to APIError — the connection/API failures that RecognizeStream emits and the session counts — instead of any exception, and move stream creation into the retry scope so a connection failure during setup is retried too. Programming errors and malformed events now fail loud instead of looping forever.
theomonnom
reviewed
Jul 17, 2026
| # Mirrors turn_detector.base.MAX_HISTORY_TURNS for tracing | ||
| _EOU_MAX_HISTORY_TURNS = 6 | ||
| # backoff before recreating the stt stream after an unrecoverable error | ||
| _STT_RECONNECT_INTERVAL = 0.5 |
Member
There was a problem hiding this comment.
Should we use an exponential backoff?
theomonnom
reviewed
Jul 17, 2026
theomonnom
approved these changes
Jul 17, 2026
Centralize the session closing check (_closing_task or _closing) behind AgentSession._is_closing and pass the bound method to the STT pipeline instead of an inline lambda. Also clarify why the STT reconnect interval stays fixed rather than backing off exponentially.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
The STT pipeline is reused across agent handoffs but keeps the node bound to the agent that created it. After a close-based handoff that agent's activity is torn down, so the new recreation loop would call the stale node, hit _get_activity_or_raise (RuntimeError, not an APIError), stop the pump, and leave the agent permanently deaf while the session stays open. Rebind the reused pipeline to the current activity's node so recreation always targets a live agent.
This was referenced Sep 14, 2026
u9g
added a commit
to livekit/agents-js
that referenced
this pull request
Sep 16, 2026
Port of livekit/agents#6418. Unlike the LLM and TTS streams, which are recreated every turn, the STT stream is long-lived: it is created once per activity and iterated by a single pump task. When it exhausted its retries it emitted an unrecoverable stt_error and the pump ended, so nothing recreated it, and because stt_error had none of the session-level tolerance that llm_error/tts_error have, the AgentSession closed immediately on the first one. - SpeechStream keeps the error that ended its retry loop as terminalError, and the default sttNode errors its output stream with it, so the pipeline can tell an exhausted stream from a closed input. - STTPipeline recreates the STT stream after a backoff on a connection failure; any other error still stops the pump. A reused pipeline is rebound to the new activity's node on handoff, and no stream is recreated once the session is closing. - AgentSession applies maxUnrecoverableErrors to stt_error as the circuit breaker; the count resets on a user transcript, mirroring how llm/tts reset on agent speaking.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5665.
Unlike the LLM and TTS streams, which are recreated every turn, the STT stream is long-lived: it is created once per agent activity and iterated by a single pump task. When it exhausted its per-utterance retries it emitted an unrecoverable
stt_errorand the pump ended, so nothing recreated it — and becausestt_errorhad none of the session-level tolerance thatllm_error/tts_errorhave, the session closed immediately on the first unrecoverable STT error.This makes the long-lived STT stream self-healing and gives it the same session-level tolerance:
_STTPipelinerecreates the STT stream (with a short backoff) after an unrecoverable error, so the agent keeps hearing the user if the provider recovers.AgentSessionappliesmax_unrecoverable_errorstostt_erroras the circuit breaker: each failed stream increments the count, and it resets on a user transcript (STT's success signal, mirroring how llm/tts reset on agent speaking).Supersedes #5666, #5997, and #6001