# Location-Relative Attention Mechanisms For Robust Long-Form Speech Synthesis

arXiv: https://arxiv.org/abs/1910.10288 · HTML: https://mlsanity.com/paper/1910.10288/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2019.

## Built on by (most cited first, 30 of 44)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data](https://mlsanity.com/paper/2402.08093/) (2024)
- [Translatotron 2: High-quality direct speech-to-speech translation with voice preservation](https://mlsanity.com/paper/2107.08661/) (2021)
- [One TTS Alignment To Rule Them All](https://mlsanity.com/paper/2108.10447/) (2021)
- [VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment](https://mlsanity.com/paper/2406.07855/) (2024)
- [Learning to Dub Movies via Hierarchical Prosody Models](https://mlsanity.com/paper/2212.04054/) (2022)
- [V2C: Visual Voice Cloning](https://mlsanity.com/paper/2111.12890/) (2021)
- [High Quality Streaming Speech Synthesis with Low, Sentence-Length-Independent Latency](https://mlsanity.com/paper/2111.09052/) (2021)
- [Modeling Concentrated Cross-Attention for Neural Machine Translation with Gaussian Mixture Model](https://mlsanity.com/paper/2109.05244/) (2021)
- [AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Person](https://mlsanity.com/paper/2108.04325/) (2021)
- [Integrated Speech and Gesture Synthesis](https://mlsanity.com/paper/2108.11436/) (2021)
- [CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversations](https://mlsanity.com/paper/2404.06690/) (2024)
- [Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speech](https://mlsanity.com/paper/2106.12896/) (2021)
- [Speaker Generation](https://mlsanity.com/paper/2111.05095/) (2021)
- [VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attention](https://mlsanity.com/paper/2102.06431/) (2021)
- [Prosodic Clustering for Phoneme-level Prosody Control in End-to-End Speech Synthesis](https://mlsanity.com/paper/2111.10177/) (2021)
- [Diffuse or Confuse: A Diffusion Deepfake Speech Dataset](https://mlsanity.com/paper/2410.06796/) (2024)
- [VoiceShop: A Unified Speech-to-Speech Framework for Identity-Preserving Zero-Shot Voice Editing](https://mlsanity.com/paper/2404.06674/) (2024)
- [Review of end-to-end speech synthesis technology based on deep learning](https://mlsanity.com/paper/2104.09995/) (2021)
- [ParaTTS: Learning Linguistic and Prosodic Cross-sentence Information in Paragraph-based TTS](https://mlsanity.com/paper/2209.06484/) (2022)
- [DeepAudio-V1:Towards Multi-Modal Multi-Stage End-to-End Video to Speech and Audio Generation](https://mlsanity.com/paper/2503.22265/) (2025)
- [Robust and Unbounded Length Generalization in Autoregressive Transformer-Based Text-to-Speech](https://mlsanity.com/paper/2410.22179/) (2024)
- [Regotron: Regularizing the Tacotron2 architecture via monotonic alignment loss](https://mlsanity.com/paper/2204.13437/) (2022)
- [Controllable speech synthesis by learning discrete phoneme-level prosodic representations](https://mlsanity.com/paper/2211.16307/) (2022)
- [Learning the joint distribution of two sequences using little or no paired data](https://mlsanity.com/paper/2212.03232/) (2022)
- [UnifySpeech: A Unified Framework for Zero-shot Text-to-Speech and Voice Conversion](https://mlsanity.com/paper/2301.03801/) (2023)
- [Enhancing audio quality for expressive Neural Text-to-Speech](https://mlsanity.com/paper/2108.06270/) (2021)
- [Zero-Shot Long-Form Voice Cloning with Dynamic Convolution Attention](https://mlsanity.com/paper/2201.10375/) (2022)
- [An investigation of phrase break prediction in an End-to-End TTS system](https://mlsanity.com/paper/2304.04157/) (2023)
- [Direct Speech-to-speech Translation without Textual Annotation using Bottleneck Features](https://mlsanity.com/paper/2212.05805/) (2022)

## Abstract (arXiv)

Despite the ability to produce human-level speech for in-domain text, attention-based end-to-end text-to-speech (TTS) systems suffer from text alignment failures that increase in frequency for out-of-domain text. We show that these failures can be addressed using simple location-relative attention mechanisms that do away with content-based query/key comparisons. We compare two families of attention mechanisms: location-relative GMM-based mechanisms and additive energy-based mechanisms. We suggest simple modifications to GMM-based attention that allow it to align quickly and consistently during training, and introduce a new location-relative attention mechanism to the additive energy-based family, called Dynamic Convolution Attention (DCA). We compare the various mechanisms in terms of alignment speed and consistency during training, naturalness, and ability to generalize to long utterances, and conclude that GMM attention and DCA can generalize to very long utterances, while preserving naturalness for shorter, in-domain utterances.