[Rehansanjay]

baseten, murf, neuphonic, resemble, telnyx and xai have no liveness check on their TTS websocket.

If the socket goes half-open after connecting — the provider, or a NAT in between, gone without a FIN or an RST — the text is pushed, no audio comes back, nothing raises, and the agent is silently mute for the rest of the call. tts.FallbackAdapter is never told.

Rewritten on top of #8123

This PR previously put a deadline on the receive itself. #8123 landed a better answer: utils.aio.WebSocketMonitor plus a liveness_timeout connect option, now used by every streaming STT plugin.

That helper is not STT-specific, and the TTS side has not adopted it. So this is now the same change for TTS — connect with autoping=False, build one monitor per connection, read through it:

async def _recv_task(ws: aiohttp.ClientWebSocketResponse) -> None:
    monitor = utils.aio.WebSocketMonitor(
        ws, liveness_timeout=self._conn_options.liveness_timeout
    )
    while True:
        msg = await monitor.receive()

baseten and telnyx read with async for msg in ws, which cannot carry a monitor, so both become explicit loops. Their existing close and error branches are unchanged.

Measured

A local server accepts the websocket upgrade and then sends nothing and answers no pings. Default liveness_timeout of 6.0:

plugin	before	after
baseten	never noticed (75s)	7.6s
murf	never noticed (75s)	6.2s
neuphonic	never noticed (75s)	6.1s
resemble	never noticed (75s)	6.1s
telnyx	never noticed (75s)	6.1s
xai	never noticed (75s)	6.1s

And the case that decides whether the mechanism is the right one — a peer that answers pings but returns no audio, i.e. a provider that is simply slow:

baseten    PASS: alive at 20.0s -- the pongs kept the timer reset
murf       PASS: alive at 20.0s
neuphonic  PASS: alive at 20.0s
resemble   PASS: alive at 20.0s
telnyx     PASS: alive at 20.0s
xai        PASS: alive at 20.0s

Text is pushed in that test, so plugins whose receive loop waits on "input sent" really do start their monitor rather than passing by not running.

That contrast is the argument for the monitor over a plain receive deadline, and it is the flaw the review found in the earlier revision here: a passive deadline cannot tell a slow provider from a dead one, so it would cut off a healthy stream waiting on a slow first token. Probing distinguishes them. It also means the input_sent_event gate the earlier revision added to baseten is no longer needed, so that is gone.

37 tests across the existing suites for these plugins pass. ruff check and ruff format --check clean.

Not included

• soniox — its _recv_loop runs for the whole life of a persistent connection, including between requests, and its except Exception catches the timeout and fails every pending request. Bounding that read needs a deadline scoped to an outstanding request, which is a larger change. Happy to take it separately.
• inworld — has receive(timeout=60.0) whose handler continues, so it polls rather than bounding anything, and the hardcoded 60 ignores the connect options. That is a behaviour change rather than a missing check.
• respeecher — rejects any non-wss:// url before opening a socket, so it could not be pointed at a local server and I could not measure it. Reading the source it does pass a timeout to ws.receive(), so it looks bounded, but that is a source read rather than a measurement.

Probe, per-plugin construction table and raw numbers: https://github.com/Rehansanjay/livekit-tts-halfopen

[Rehansanjay]

Converting this to a draft. The automated review found three problems and all three are real — the measurement that motivated this holds, but the fix is wrong, and it would trade a rare failure for a common one.

The deadline starts before any text is sent. This is the one that matters. In baseten, _send_task waits on self._input_ch for text from the LLM while _recv_task begins reading immediately, and both run concurrently:

async def _send_task(ws):
    async for data in self._input_ch:      # waits for the LLM
        ...
async def _recv_task(ws):
    async for msg in ws:                   # deadline already running

With ws_receive=conn_options.timeout, a slow first token — a long tool call, a slow model, a SynthesizeStream held open across a turn — expires the read before the provider has been asked for anything, and a healthy stream fails. neuphonic can expire before tokenization emits a sentence and murf between tokens, for the same reason.

I argued the opposite in the description above: "ws_receive only fires when nothing arrives for the whole timeout — which, for a stream that has been handed text, is already a failure." That is wrong, because the stream has not been handed text yet when the clock starts. A dead socket is rare; a slow first token is not, so as written this is a net regression.

Pooled sockets keep the first request's deadline. murf and neuphonic go through ConnectionPool, and ws_receive is fixed when the socket is created. A socket opened for a 10s request and later handed to a 30s one still enforces 10s, and prewarmed sockets carry whatever value was in scope at prewarm. A per-request value does not belong on a per-connection object.

soniox's idle persistent connection expires. Its _recv_loop runs for the whole life of the connection, including between requests, so a prewarmed idle socket is closed after timeout seconds of normal quiet and the next stream has to reconnect.

What this means

A connection-level ws_receive cannot distinguish "waiting for input" from "waiting for audio after input", and for a streaming TTS that distinction is the whole thing. cartesia, deepgram and smallestai avoid it by calling ws.receive(timeout=...) at the point where they are actually expecting a response, rather than setting it as a socket property — which is the difference I missed when I took their behaviour as precedent for this form.

So the deadline has to be armed after the first send and reset on each frame, which is a per-plugin change to the receive loops rather than one line at connect, and for soniox it also has to be disarmed between requests.

That is a bigger change than this PR, and where the deadline should live looks like a maintainer's call rather than mine — in the base class alongside conn_options, or per plugin. Happy to implement whichever you prefer; I would rather ask than ship seven risky receive-loop rewrites.

The underlying problem is unchanged and still worth fixing. Seven plugins — baseten, murf, neuphonic, telnyx, resemble, soniox, xai — never notice a half-open socket at all, measured at a 75-second watch against a server that accepts the upgrade and then sends nothing and answers no pings. The agent goes mute for the rest of the call and FallbackAdapter is never told, because nothing raises. Numbers and probe: https://github.com/Rehansanjay/livekit-tts-halfopen

Thanks for the review — the second finding in particular saved a regression that my own test setup would not have caught, because the probe pushes text immediately and so never exercises a slow first token.

[devin-ai-integration]

Devin Review found 1 new potential issue.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

[devin-ai-integration]

🔴 Silent provider leaves speech stream waiting

When a provider stops sending audio but answers pings, monitor.receive() waits indefinitely. WebSocketMonitor resets its deadline on every PONG, so the stream never reaches fallback.

Learn more

The receive loop now checks transport activity rather than waiting for application audio. WebSocketMonitor.receive consumes PONG frames and resets its timer each time. A reachable provider can keep answering pings while its synthesis pipeline stalls, so no audio arrives and the receive loop never fails. The same replacement appears in the Murf, NeuPhonic, Resemble, Telnyx, and xAI streams.

Example: A provider accepts the text, stops producing audio, and answers each ping. With timeout=10 and liveness_timeout=6, the stream remains blocked past ten seconds because each PONG renews the monitor's six-second deadline.

Recommended fix: Keep the monitor for detecting dead connections, but add a separate per-request audio or synthesis-progress deadline in the receive loops. Start it when text is sent for providers that can remain idle before input; do not let PONG frames renew it.

---

Was this helpful? React with 👍 or 👎 to provide feedback.