Repository navigation
feat(deepgram): support speed parameter in Flux TTS - #7483
Conversation
Follow-up to livekit#7480 for TTSv2 (/v2/speak). Flux takes `speed` from 0.5 to 1.5 in 0.05 steps on both the WebSocket and the REST endpoint; the plugin never sent it. Add `speed` to `TTSv2.__init__` and `update_options`, check the range and the step up front, and send it on both the streaming (WebSocket URL) and the synthesize (HTTP) path when set, the same way as in the v1 TTS.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if self._opts.speed is not None: | ||
| config["speed"] = self._opts.speed |
There was a problem hiding this comment.
🔴 Existing Flux streams change speaking rate
When update_options changes speed before a pending stream connects, _connect_ws uses the new speed instead of the stream's snapshot. An active stream can also change rate between segments after pool invalidation.
Learn more
A Flux stream copies its options when constructed in SynthesizeStreamv2, but the shared connection callback reads the TTS instance's current speed. A stream that has not acquired a socket yet therefore uses a later speed. A stream already running can acquire another socket for its next flushed text segment, and pool invalidation forces that new socket to use the later speed. The emitter and other stream settings still use the original snapshot.
Example: Create a stream at speed 0.7, then call update_options(speed=1.5) before its first text segment. Its WebSocket requests 1.5 even though that stream captured 0.7; a stream spanning the update can speak its first and second segments at different rates.
Recommended fix: Pass the stream's copied connection parameters to the pool acquisition or otherwise bind each stream to a consistent speed across its segments. Ensure invalidation still affects new streams without changing the settings of existing ones.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
This matches how the other connection params (model, encoding, sample_rate, bit_rate) already work here and in the v1 TTS (#7480): they are baked into the pooled socket, and update_options invalidates the pool so the next socket picks up the new value. For speed, picking up the new rate at the next segment is what a caller of update_options(speed=...) expects. Binding each stream to its own socket params would change how the pool works for every option, so I would keep it out of this PR.
A speed within the step tolerance but not exactly on it (1.00000001, or 1.5000000000000002 from float arithmetic) passed the check and was sent as is, so Deepgram rejected it. Snap it to the step before checking the range and store that value, so both request paths send e.g. 1.5.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Check the range, with a little slack for float noise, before snapping to the 0.05 step, so inf, nan and huge values raise ValueError instead of OverflowError.
Follow-up to #7480, for Flux TTS (
TTSv2,/v2/speak).Flux takes a
speedparameter from 0.5 to 1.5 in 0.05 steps (docs) on bothwss://…/v2/speakandPOST /v2/speak, butTTSv2never sent it.This adds
speedtoTTSv2.__init__andupdate_optionsand sends it on both paths when set, the same way as in the v1TTS:stream()(WebSocket): added to the connection URL;update_options(speed=...)invalidates the pool like the other URL params.synthesize()(HTTP): added to the request params.None, which leaves it out of the request.ValueErrorup front. The API rejects both with a 400, but on the WebSocket path that only surfaces as a failed handshake.Tested against the live API (
flux-alexis-en, the same sentence, speech length without the trailing silence Flux adds, median of 3 runs):speedsynthesize()stream()update_options(speed=1.5)on a TTS with a pooled socket took the next streams from 3.58 s to 2.20 s. 0.45, 1.55 and 1.12 return 400 from the API.Unit tests cover both request paths, the default, the range and step checks and pool invalidation.
Flux can also change speed on an open socket with a
Configuremessage, without reconnecting. I kept to the URL and pool invalidation to match the other options; happy to switchupdate_optionstoConfigureif you'd prefer that.