Repository navigation
feat: register local ASR as a pluggable stt engine - #620
Open
TSOFTP-afk wants to merge 1 commit into
Open
TSOFTP-afk wants to merge 1 commit into
TSOFTP-afk wants to merge 1 commit into
Conversation
- move the SenseVoice/FunASR engine out of server/asr_server.py into a registry plugin (stt/funasr.py); registry.py presets category "stt" since the 2026-03-31 modularization but had no implementation - keep the /api/asr WebSocket protocol byte-identical and preserve the silent degradation when funasr is not installed - add --asr config key to select the STT engine (default: funasr) - fix stale tts help text (cosyvoice/fishtts/indextts2 were removed) and the unused "output" registry preset (actual category: streamout) - migrate tests to the plugin layout, add a /api/asr protocol contract test; document local ASR in README.md / README-EN.md / docs/api.md Refs lipku#601, lipku#604
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
registry.pyhas preset a"stt"category since the 2026-03-31 modularization refactor, but it never received an implementation: zero plugins, zerocreate("stt", ...)call sites. Meanwhile the local SenseVoice/FunASR engine (#609, for #604) lives hard-coded insideserver/asr_server.py, so the engine cannot be selected or replaced without editing the WebSocket handler.This PR completes that design: STT becomes a registry plugin, and FunASR is the first registered engine.
What changed
stt/base_stt.py(new) —BaseSTT:ensure_loaded()/transcribe(audio_bytes, sample_rate, is_final)/is_available().stt/funasr.py(new) —@register("stt", "funasr")FunASRSTT. The engine is moved verbatim fromserver/asr_server.py: lazy singleton,asyncio.Lock+threading.Lockpair, mutex-serialized inference,iic/SenseVoiceSmall+fsmn-vad,vad_kwargs={"max_single_segment_time": 30000},trust_remote_code, <20 ms utterances skipped,rich_transcription_postprocess.server/asr_server.py— keeps the/api/asrWebSocket protocol byte-identical; the engine is now obtained throughregistry.create("stt", ...). Silent degradation is preserved: whenfunasris not installed, the endpoint is simply not registered.config.py/config.yaml— new--asrkey (defaultfunasr). Also fixes the stale tts help text (cosyvoice/fishtts/indextts2were removed from the codebase but were still listed in the help strings).registry.py— the unused"output"preset is corrected to"streamout"(the category output plugins actually register under).tests/test_asr_server.py— the two behavior tests (lazy singleton constructed once under concurrency; shared-model inference serialized) are migrated to the plugin, and a new protocol contract test locks the/api/asrmessage flow: config JSON → binary PCM16 frames →{"is_speaking": false}→ reply{"text", "mode", "is_final", "timestamp"}. Tests remain dependency-free (fake modules injected intosys.modules).README.md/README-EN.md/docs/api.md, with an explicit note thatserver/asr_server.py(user speech recognition) andavatars/audio_features/base_asr.py(lip-sync audio features) are unrelated despite the similar names.Not changed
/api/asrwire protocol, theweb/asr/frontend, the avatar/TTS/LLM pipelines, andavatars/audio_features/*(lip-sync feature extractors).Verification
Engine latency, measured to document the feature (the engine itself is untouched; same environment for all runs — Windows, RTX 3060 Laptop 6 GB, torch 2.13.0+cu130, funasr 1.4.16, 8.3 s utterance @ 16 kHz, 7 timed runs after one warm-up):
The run-to-run spread is environment noise (GPU clocks/thermals), not a code difference — the engine code is identical before and after the refactor. The protocol contract test is the behavior-preservation evidence.
Refs #601, #604