mlsanityData to 7 Oct 2026

Fine-grained Noise Control for Multispeaker Speech Synthesis

arXiv 2204.05070 ↗First posted 11 Apr 2022cs.SD cs.CL cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

A text-to-speech (TTS) model typically factorizes speech attributes such as content, speaker and prosody into disentangled representations.Recent works aim to additionally model the acoustic conditions explicitly, in order to disentangle the primary speech factors, i.e. linguistic content, prosody and timbre from any residual factors, such as recording conditions and background noise.This paper proposes unsupervised, interpretable and fine-grained noise and prosody modeling. We incorporate adversarial training, representation bottleneck and utterance-to-frame modeling in order to learn frame-level noise representations. To the same end, we perform fine-grained prosody modeling via a Fully Hierarchical Variational AutoEncoder (FVAE) which additionally results in more expressive speech synthesis.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 19 of its 29 references that are in the map. First posted on arXiv 11 Apr 2022.

Builds on

19 of 29 references.

References · 19

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  5. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  6. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  7. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  8. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  9. 2017Generalized End-to-End Loss for Speaker Verificationcited by 290
  10. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
  11. 2018LPCNet: Improving Neural Speech Synthesis Through Linear Predictioncited by 109
  12. 2020End-to-End Adversarial Text-to-Speechcited by 77
7 more
  1. 2020Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modelingcited by 65
  2. 2020Sudo rm -rf: Efficient Networks for Universal Audio Source Separationcited by 50
  3. 2020Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesiscited by 41
  4. 2020Attentron: Few-Shot Text-to-Speech Utilizing Attention-Based Variable-Length Embeddingcited by 27
  5. 2021High Quality Streaming Speech Synthesis with Low, Sentence-Length-Independent Latencycited by 19
  6. 2021STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speechcited by 18
  7. 2020DenoiSpeech: Denoising Text to Speech with Frame-Level Noise Modelingcited by 13

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 1.

2 papers.

  1. 2022NoreSpeech: Knowledge Distillation based Conditional Diffusion Model for Noise-robust Expressive TTScited by 8
  2. 2024Noise-robust zero-shot text-to-speech synthesis conditioned on self-supervised speech-representation model with adapterscited by 4