Click here to download the project base paper text to the speech.
Abstract:
Single-stage text-to-speech models have been actively studied recently, and their results have outperformed two-stage pipeline systems. Although the previous single-stage model has made great progress, there is room for improvement in terms of its intermittent unnaturalness, computational efficiency, and strong dependence on phoneme conversion. In this work, we introduce VITS2, a single-stage text-tospeech model that efficiently synthesizes a more natural speech by improving several aspects of the previous work. We propose improved structures and training mechanisms and present that the proposed methods are effective in improving naturalness, similarity of speech characteristics in a multi-speaker model, and efficiency of training and inference. Furthermore, we demonstrate that the strong dependence on phoneme conversion in previous works can be significantly reduced with our method, which allows a fully end-to-end single-stage approach.
Introduction Recent developments in deep neural network-based text-tospeech have seen significant advancements. Deep neural network-based text-tospeech is a method for generating corresponding raw waveforms from input texts; it has several interesting features that often make the text-tospeech task challenging. A quick review of the features reveals that the text-to-speech task involves converting text, which is a discontinuous feature, into continuous waveforms. The input and output have a time step difference of hundreds of times, and the alignment between them must be very precise to synthesize high-quality speech audio.
