Repository navigation
Inference benchmarks on CPU / single-thread performance #5
Description
Hi @jaywalnut310 ,
Many thanks for your work!
As usual this is very thorough, open and inspiring.
In your paper you publish the GPU speed benchmarks:
We measured the synchronized elapsed time over the entire process to generate raw waveforms from phoneme sequences with 100 sentences randomly selected from the test set of the LJ Speech dataset. We used a single NVIDIA V100 GPU with a batch size of 1.
But somehow you do not say anything about running on CPU (1 CPU thread). Notably, this is also omitted from papers like TalkNet, / FastSpeech / GlowTTS (I believe this paper is mostly GlowTTS meets HifiGAN).
The only paper saying anything about CPU speed is LightSpeech:
Is this because flow-based models do not lend themselves well to CPU inference?
Activity
jaywalnut310 commentedon Jun 15, 2021
Hi @snakers4. We reported inference speed tests on a GPU server rather than CPU only environments, as it is a representative indicator for speed comparison in many papers. I think there is no reason for VITS and flow-based TTS models to be worse than other TTS models such as FastSpeech 2, but If you're focusing on CPU or on-device inference, it would be great to check each model's real time factor.
snakers4 commentedon Jun 17, 2021
Many thanks for your reply!
Did you by any chance run any benchmarks of your models on CPU only environments that you did not include in the final paper?
We are still using plain tacotron as a main TTS models and basically now we are weighting the pros and cons of investing time in developing something something akin to TalkNet / LightSpeech / FastSpeech (there is no good code available for smaller models though) or your new pipeline.
TalkNet / LightSpeech / FastSpeech pros and cons:
VITS pros and cons:
Correct me if I am wrong here, but I believe that many people do similar mental calculations now.