[GregHolmes]

Summary

RecognizeStream._main_task gives a stream max_retry attempts (3 by default). _num_retries only ever increments, and its single reset lives in _metrics_monitor_task, on FINAL_TRANSCRIPT:

https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/stt/stt.py#L541-L543

That makes the budget a stream-lifetime counter replenished by the caller speaking, rather than a consecutive-failure counter replenished by the connection recovering.

A stream that reconnects cleanly but sits through silence never earns its budget back. max_retry drops spread over one long call then kill it permanently, with recoverable=False and no further reconnect, even though every one of those reconnects succeeded. The drops need not be consecutive or close together: a blip at minute 1, 10 and 40, then a fourth, is enough.

Why silence is the ordinary case, not an edge case

The gaps where a voice agent has no final transcript are exactly the gaps where it is most exposed: the agent is talking, the caller is on hold, an IVR is playing, someone stepped away. That is also when an idle socket is most likely to be reaped by a NAT or a load balancer. The condition that spends the budget and the condition that blocks its reset are the same condition.

Reproduction

Three streams whose connections all succeed and then drop. Ten drops each. The only difference is what they deliver in between.

what the stream delivers between drops	connections made	outcome
nothing at all	4	dies: failed to recognize speech after 3 attempts
a FINAL_TRANSCRIPT each time	11	survives
RECOGNITION_USAGE only	4	dies

The third row is the ordinary case: the Deepgram plugin emits a usage event every 5s while audio flows, speech or no speech. Same failures, opposite outcome, decided purely by whether somebody was talking.

Instrumenting _num_retries shows the mechanism. With transcripts it sits at a steady 1 forever, reset by the event and incremented by the failure. Without them it climbs 0, 1, 2, 3 and the stream is gone.

A runnable script is in the PR below as two tests, but the shape is just:

class _FlappingStream(RecognizeStream):
    async def _run(self) -> None:
        self.runs += 1
        if self._emit is not None:
            self._event_ch.send_nowait(self._emit)   # usage event, no transcript
            await asyncio.sleep(0)
        if self.runs <= self._drops:
            raise APIConnectionError("socket dropped")
        await asyncio.sleep(3600)

Is this intended?

I want to flag that it might be. "A stream that has never been understood should not retry forever" is a defensible rule, and resetting on FINAL_TRANSCRIPT implements it. If that is the intent, then the bug is narrower: the budget should still reset once a reconnect succeeds, rather than only once recognition succeeds.

Suggested fix

Reset on any event from the provider rather than only on FINAL_TRANSCRIPT. Any event proves the connection came up and delivered. A connection that has never delivered anything at all still gives up as before, so this does not become an unbounded retry.

Reported as #7208.

Happy to take it a different way if you would rather the reset hung off a successful connection explicitly, though that needs a signal out of _run() that the base class does not have today.

🤖 Generated with Claude Code

[longcw]

thanks for the pr, I think the analysis is right but need some changes:

[longcw]

I'd like to put the reset back in _metrics_monitor_task and widen it from FINAL_TRANSCRIPT to other events that verify the connection is health , instead of overriding send_nowait in _HealthySignallingChan.

[longcw]

also need to add that for MultiSpeakerAdapter  

agents/livekit-agents/livekit/agents/stt/multi_speaker_adapter.py

Lines 143 to 145 in 4de6232

	async for ev in event_aiter:
	if ev.type == SpeechEventType.FINAL_TRANSCRIPT:
	self._num_retries = 0

[GregHolmes]

Devin raised two valid edge cases: usage events can be generated locally, and the monitor can run too late. Would you be comfortable with a small per-attempt recovery marker, set only after a real provider response and checked by the retry logic? That avoids both endless retries and missed recoveries.

[GregHolmes]

Addressed in e633a3f.

• Removed the custom event-channel override and restored the reset to _metrics_monitor_task.
• The budget now resets after provider-originated INTERIM_TRANSCRIPT, PREFLIGHT_TRANSCRIPT, FINAL_TRANSCRIPT, or RECOGNITION_USAGE events. START_OF_SPEECH and END_OF_SPEECH stay excluded because adapters can synthesize them locally.
• Applied the same reset conditions in MultiSpeakerAdapterWrapper.
• Updated regression coverage for all health-event types and the multi-speaker adapter usage-event path.

Verified with uv run pytest tests/test_stt_base.py tests/test_multi_speaker_adapter.py and Ruff.

[devin-ai-integration]

Devin Review found 2 new potential issues.

[devin-ai-integration]

🔴 Local usage defeats retry limits

Locally generated RECOGNITION_USAGE resets _num_retries without a provider response. Deepgram cleanup emits it after each dropped connection, enabling endless retries.

Learn more

RECOGNITION_USAGE is a framework event, not necessarily a server acknowledgement. Several plugins derive it from locally pushed audio. Deepgram also flushes connection-lifetime usage while unwinding a failed _run; that event reaches this monitor before the retry delay completes. The monitor clears the counter, then _main_task increments it to one, so repeated connection failures never reach max_retry. The same event list in MultiSpeakerAdapterWrapper creates the same false recovery signal for the outer stream.

Example: Deepgram establishes a WebSocket, receives no message, and the socket closes. Its cleanup reports the short socket lifetime as usage. Each failed attempt resets _num_retries, so an outage causes reconnects indefinitely rather than stopping after three retries.

Recommended fix: Reset from an explicit successful-connection or provider-message signal. Do not treat RECOGNITION_USAGE as proof of recovery unless each plugin guarantees the event came from the provider; update MultiSpeakerAdapterWrapper consistently.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🟡 Immediate responses miss retry reset

If _run emits a qualifying event then fails without yielding, _num_retries remains stale during error handling. The final allowed attempt then terminates despite delivering a response.

Learn more

The event producer and metrics monitor are separate tasks. send_nowait only queues an event; it does not run the monitor. If _run raises immediately afterward, _main_task can inspect _num_retries == max_retries and terminate before the reset executes. The new _FlappingStream avoids this ordering by calling await asyncio.sleep(0) after every emitted event, but RecognizeStream imposes no such requirement on plugins.

Example: With _num_retries == 3, _run calls self._event_ch.send_nowait(usage) and immediately raises APIConnectionError. _main_task sees three retries and raises the fatal wrapper error. The monitor processes usage only after termination, although the response was intended to replenish the budget.

Recommended fix: Make response observation synchronous with retry accounting rather than mutating _num_retries from the monitor task. For example, track an attempt generation and a response marker that _main_task can inspect reliably before classifying the failure.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[davidzhao]

Thanks @GregHolmes for this! #8123 now resets the STT retry budget whenever the provider shows progress, which covers this, so closing.