Repository navigation
feat(gradium): optional VAD-based start of speech in STT - #8247
Merged
Merged
Conversation
Add vad_start_threshold. When set, a step whose VAD inactivity probability drops below it starts speech before any text is transcribed. A turn only opens after silence was observed, since the estimate reads low before the model has heard any audio. Off by default.
The server sends a few steps before any audio is processed (total_duration_s == 0) whose estimate reads as activity. Ignoring those lets a user who talks from the very start open a turn on the VAD, where waiting for silence first could miss it and fall back to the first word.
davencyw
marked this pull request as ready for review
October 29, 2026 14:49
The check tested vad_bucket for truthiness, so bucket 0 (the 0.5s horizon) silently turned the feature off. Only None disables it.
The end-of-turn check still tested vad_bucket for truthiness, so with bucket 0 a turn opened by the VAD start of speech never closed. Same change as in livekit#7871.
Contributor
Author
|
@longcw @tinalenguyen could one of you take a look when you have a moment? A customer asked for this: fast barge-in on Gradium STT without adding a local VAD. It's opt-in ( |
longcw
approved these changes
Nov 3, 2026
Contributor
|
could you resolve the conflicts before we can merge? |
…-of-speech # Conflicts: # tests/test_plugin_gradium_stt.py
Contributor
Author
|
Done, merged main and resolved the conflicts (both sides only added tests). Thanks for the review! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an optional
vad_start_thresholdto the Gradium STT. When set, the server VAD instepmessages starts speech as soon as the inactivity probability drops below it, instead of waiting for the first transcribed word. This allows fast barge-in without a local VAD, like Gradbot and Unmute (which use 0.4). Off by default.total_duration_s == 0) are ignored: their estimate reads as activity whatever the input, so they would open a false turn on every new connection.Checked against the real server, speech start detected after speech onset:
vad_start_threshold=0.4No false starts on 10s of near silence or 6s of white noise.
With
vad=Noneandturn_detection="stt", this start of speech sets the user state to speaking right away, so an agent can barge in on it:Note: this overlaps with #7870 and #7871 (same file), will rebase on whichever lands first.