Repository navigation
fix(stt): detect half-open websockets in OpenAI, ElevenLabs, and Inworld STT plugins - #1
Closed
ninglinLiu wants to merge 4 commits into
Closed
ninglinLiu wants to merge 4 commits into
ninglinLiu wants to merge 4 commits into
Conversation
…5, Fable 5.1 and Mythos 5.1 (livekit#7511)
…rld STT plugins Add heartbeat=30.0 to ws_connect and handle WSMsgType.ERROR by raising retryable APIConnectionError in OpenAI realtime, ElevenLabs, and Inworld STT plugins. Without heartbeat, a silently dropped socket (half-open TCP connection, no FIN/RST) is never noticed: recv_task parks on ws.receive() forever and the reconnect loop, which is exception-driven, never runs. The heartbeat closes the socket when a ping goes unanswered, and that surfaces as WSMsgType.ERROR. ws.exception() is the only place the reason survives, so we chain it to the APIConnectionError. Add regression tests for each plugin proving they notice a silent websocket via heartbeat and raise retryable APIConnectionError within bounded time. Fixes livekit#7900 Co-authored-by: NingLin Liu <x07514475479@gmail.com>
Run ruff format on 4 files per upstream CI requirements: - livekit-plugins-elevenlabs/livekit/plugins/elevenlabs/stt.py - tests/test_plugin_openai_stt_halfopen.py - tests/test_plugin_elevenlabs_stt_halfopen.py - tests/test_plugin_inworld_stt_halfopen.py Restructured the Inworld test's _stream function to keep the type: ignore[arg-type] comment on the http_session argument line where mypy will apply it, while satisfying ruff format requirements. Co-authored-by: NingLin Liu <x07514475479@gmail.com>
Owner
Author
|
Superseded by upstream livekit#7912. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
OpenAI, ElevenLabs, and Inworld streaming STT plugins never notice a half-open websocket (provider/NAT gone without FIN/RST). They wait forever on
ws.receive(), so FallbackAdapter never fails over. Deepgram, Soniox, and similar plugins already have fixes that passheartbeat=30.0tows_connectand mapWSMsgType.ERRORto retryableAPIConnectionError(see merged fixes like livekit#7206).Solution
This PR applies the same pattern to OpenAI realtime STT, ElevenLabs STT, and Inworld STT:
Added
heartbeat=30.0tows_connectin all three pluginsHandle
WSMsgType.ERRORby raisingAPIConnectionErrorAdded regression tests for each plugin proving they:
Testing
All new tests pass:
tests/test_plugin_openai_stt_halfopen.py✅tests/test_plugin_elevenlabs_stt_halfopen.py✅tests/test_plugin_inworld_stt_halfopen.py✅Existing unit tests still pass with no regressions.
Linter (ruff) checks pass for all modified files.
Changes
Modified files:
livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/stt.pylivekit-plugins/livekit-plugins-elevenlabs/livekit/plugins/elevenlabs/stt.pylivekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/stt.pyNew test files:
tests/test_plugin_openai_stt_halfopen.pytests/test_plugin_elevenlabs_stt_halfopen.pytests/test_plugin_inworld_stt_halfopen.pyFixes livekit#7900
Note: This fix follows the same pattern as the merged Deepgram/Soniox/etc fixes. I kept the heartbeat inline in each plugin rather than creating a shared helper, as suggested in the issue discussion, to avoid expanding the scope.