mlsanityData to 7 Oct 2026

Multi-Scale Spectrogram Modelling for Neural Text-to-Speech

arXiv 2106.15649 ↗First posted 29 Jun 2021eess.AS cs.LG cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose a novel Multi-Scale Spectrogram (MSS) modelling approach to synthesise speech with an improved coarse and fine-grained prosody. We present a generic multi-scale spectrogram prediction mechanism where the system first predicts coarser scale mel-spectrograms that capture the suprasegmental information in speech, and later uses these coarser scale mel-spectrograms to predict finer scale mel-spectrograms capturing fine-grained prosody. We present details for two specific versions of MSS called Word-level MSS and Sentence-level MSS where the scales in our system are motivated by the linguistic units. The Word-level MSS models word, phoneme, and frame-level spectrograms while Sentence-level MSS models sentence-level spectrogram in addition. Subjective evaluations show that Word-level MSS performs statistically significantly better compared to the baseline on two voices.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 27 references that are in the map. First posted on arXiv 29 Jun 2021.

Builds on

21 of 27 references.

References · 21

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  3. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  4. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  5. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  6. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  7. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 171
  8. 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
  9. 2019MelNet: A Generative Model for Audio in the Frequency Domaincited by 41
  10. 2019Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTScited by 33
  11. 2019DurIAN: Duration Informed Attention Network For Multimodal Synthesiscited by 33
  12. 2020Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody priorcited by 33
9 more
  1. 2019CHiVE: Varying Prosody in Speech Synthesis with a Linguistically Driven Dynamic Hierarchical Conditional Variational Networkcited by 25
  2. 2018Expressive Speech Synthesis via Modeling Expressions with Variational Autoencodercited by 22
  3. 2020AlignTTS: Efficient Feed-Forward Text-to-Speech System without Explicit Alignmentcited by 20
  4. 2020Controllable neural text-to-speech synthesis using intuitive prosodic featurescited by 19
  5. 2020Hierarchical Prosody Modeling for Non-Autoregressive Speech Synthesiscited by 16
  6. 2020CAMP: a Two-Stage Approach to Modelling Prosody in Contextcited by 12
  7. 2019Dynamic Prosody Generation for Speech Synthesis using Linguistics-Driven Acoustic Embedding Selectioncited by 11
  8. 2020Prosodic Representation Learning and Contextual Sampling for Neural Text-to-Speechcited by 10
  9. 2019Traditional Machine Learning for Pitch Detectioncited by 1

Built on by

Not cited yet.