mlsanityData to 14 Oct 2026

Location-Relative Attention Mechanisms For Robust Long-Form Speech Synthesis

arXiv 1910.10288 ↗First posted 23 Oct 2019cs.CL cs.LG cs.SDcited by 44

Not on the map. Why?

Abstract (arXiv)

Despite the ability to produce human-level speech for in-domain text, attention-based end-to-end text-to-speech (TTS) systems suffer from text alignment failures that increase in frequency for out-of-domain text. We show that these failures can be addressed using simple location-relative attention mechanisms that do away with content-based query/key comparisons. We compare two families of attention mechanisms: location-relative GMM-based mechanisms and additive energy-based mechanisms. We suggest simple modifications to GMM-based attention that allow it to align quickly and consistently during training, and introduce a new location-relative attention mechanism to the additive energy-based family, called Dynamic Convolution Attention (DCA). We compare the various mechanisms in terms of alignment speed and consistency during training, naturalness, and ability to generalize to long utterances, and conclude that GMM attention and DCA can generalize to very long utterances, while preserving naturalness for shorter, in-domain utterances.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 17; 2022: 9; 2023: 3; 2024: 8; 2025: 4; 2026: 3.

44 papers.

Sort:

By area:

  1. 2021A Survey on Neural Speech Synthesiscited by 184
  2. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 75
  3. 2021Translatotron 2: High-quality direct speech-to-speech translation with voice preservationcited by 71
  4. 2021One TTS Alignment To Rule Them Allcited by 47
  5. 2024VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignmentcited by 38
  6. 2022Learning to Dub Movies via Hierarchical Prosody Modelscited by 29
  7. 2021V2C: Visual Voice Cloningcited by 24
  8. 2021High Quality Streaming Speech Synthesis with Low, Sentence-Length-Independent Latencycited by 19
  9. 2021Modeling Concentrated Cross-Attention for Neural Machine Translation with Gaussian Mixture Modelcited by 18
  10. 2021AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Personcited by 17
  11. 2021Integrated Speech and Gesture Synthesiscited by 16
  12. 2024CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversationscited by 15
32 more
  1. 2021Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speechcited by 14
  2. 2021Speaker Generationcited by 14
  3. 2021VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attentioncited by 12
  4. 2021Prosodic Clustering for Phoneme-level Prosody Control in End-to-End Speech Synthesiscited by 8
  5. 2024Diffuse or Confuse: A Diffusion Deepfake Speech Datasetcited by 8
  6. 2024VoiceShop: A Unified Speech-to-Speech Framework for Identity-Preserving Zero-Shot Voice Editingcited by 7
  7. 2021Review of end-to-end speech synthesis technology based on deep learningcited by 6
  8. 2022ParaTTS: Learning Linguistic and Prosodic Cross-sentence Information in Paragraph-based TTScited by 5
  9. 2025DeepAudio-V1:Towards Multi-Modal Multi-Stage End-to-End Video to Speech and Audio Generationcited by 5
  10. 2024Robust and Unbounded Length Generalization in Autoregressive Transformer-Based Text-to-Speechcited by 4
  11. 2022Regotron: Regularizing the Tacotron2 architecture via monotonic alignment losscited by 3
  12. 2022Controllable speech synthesis by learning discrete phoneme-level prosodic representationscited by 3
  13. 2022Learning the joint distribution of two sequences using little or no paired datacited by 3
  14. 2023UnifySpeech: A Unified Framework for Zero-shot Text-to-Speech and Voice Conversioncited by 3
  15. 2021Enhancing audio quality for expressive Neural Text-to-Speechcited by 2
  16. 2022Zero-Shot Long-Form Voice Cloning with Dynamic Convolution Attentioncited by 2
  17. 2023An investigation of phrase break prediction in an End-to-End TTS systemcited by 2
  18. 2022Direct Speech-to-speech Translation without Textual Annotation using Bottleneck Featurescited by 1
  19. 2023Energy-Based Models For Speech Synthesiscited by 1
  20. 2024Acquiring Pronunciation Knowledge from Transcribed Speech Audio via Multi-task Learningcited by 1
  21. 2021Triple M: A Practical Text-to-speech Synthesis System With Multi-guidance Attention And Multi-band Multi-time LPCNetcited by 0
  22. 2021Nana-HDR: A Non-attentive Non-autoregressive Hybrid Model for TTScited by 0
  23. 2021PAMA-TTS: Progression-Aware Monotonic Attention for Stable Seq2Seq TTS With Accurate Phoneme Duration Controlcited by 0
  24. 2022R-MelNet: Reduced Mel-Spectral Modeling for Neural TTScited by 0
  25. 2022EPIC TTS Models: Empirical Pruning Investigations Characterizing Text-To-Speech Modelscited by 0
  26. 2024RELATE: A Modern Processing Platform for Romanian Languagecited by 0
  27. 2025Towards Film-Making Production Dialogue, Narration, Monologue Adaptive Moving Dubbing Benchmarkscited by 0
  28. 2025MM-MovieDubber: Towards Multi-Modal Learning for Multi-Modal Movie Dubbingcited by 0
  29. 2025Adaptive Duration Model for Text Speech Alignmentcited by 0
  30. 2026Archon: A Unified Multimodal Model for Holistic Digital Human Generationcited by 0
  31. 2026Imitation Learning for Elder-Facing Speech Synthesiscited by 0
  32. 2026Stable Autoregressive Speech Generation with Low-Frame-Rate High-Dimensional Continuous Tokenscited by 0