VARA-TTS: Non-Autoregressive Text-to-Speech based on Very Deep VAE with Residual Attention

Abstract

Non-autoregressive (non-AR) text-to-speech (TTS) models usually contain a separate phoneme-level duration predictor, which is trained with the supervision signal generated during training or from an external aligner. This paper proposes VARA-TTS, a non-AR TTS model using a very deep Variational Autoencoder (VDVAE) with Residual Attention mechanism, which refines the textual-to-acoustic alignment layer-wisely. Hierarchical representations with different temporal resolutions from the VDVAE are used as queries for the residual attention module. By leveraging coarse global alignment from the precedent attention layer, the descendant attention layer can produce a refined version. This coarse-to-fine mechanism distributes the burden of learning the textual-to-acoustic alignment into multiple attention layers, leading to more robust generation at inference. An utterance-level speaking speed factor is computed by a jointly-trained speaking speed predictor to determine the number of acoustic frames at inference. Experimental results show that VARA-TTS achieves similar speech quality to an AR counterpart Tacotron 2 but an order-of-magnitude speed-up at inference; and outperforms analogous non-AR models, Glow-TTS and BVAE-TTS, in terms of speech quality.

Multi-speaker TTS (Chinese)

Text: 他怎么也该留下一只让我尝尝才是呀!

Text: 我回家传照片去了,请你告诉我,凤琴的号码。

Text: 李元芳大招清一波线。

Text: 据伦敦北区消防队的官员说,摩总统主张摩加入独联体。

Text: 出租车里电台广告都有宣传广告,大厦财大气粗啊!

Text: 我在外面,随便吃点就好了。

Text: 曹操还是有一技能三段位移的。

Text: 十年沧海,劈波斩浪。

Text: 广东省政协常委陈缵光走进“蚁族”居所。

Text: 在该超市的促销海报上,双汇肘花火腿”也赫然在列。

Single-speaker TTS (English)

Text: the latter raised eighteen pence among them to pay for a truss of straw for the poor woman to lie on.

Text: They were never left quite alone for fear of suicide, and for the same reason they were searched for weapons or poisons.

Text: provided only that their security was not jeopardized, and dependent upon the enforcement of another new rule,

Text: One of the earliest of the big operators in fraudulent finance was Edward Beaumont Smith,

Text: with the idea of subjecting her to the irritant poison slowly but surely until the desired effect, death, was achieved.

Text: was to be the rule for all convicted prisoners throughout the early stages of their detention;

Text: Every facility was promised. The sanction of the Secretary of State would not be withheld if plans and estimates were duly submitted,

Text: treated the gunshot wound in the left thigh.

Text: He admitted to fantasies about being powerful and sometimes hurting and killing people, but refused to elaborate on them.

Text: as to the fact that he was an outstanding man, end quote.