# Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTS

arXiv: https://arxiv.org/abs/1906.00672 · HTML: https://mlsanity.com/paper/1906.00672/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2019.

## Built on by (most cited first, 30 of 33)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [Translatotron 2: High-quality direct speech-to-speech translation with voice preservation](https://mlsanity.com/paper/2107.08661/) (2021)
- [VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment](https://mlsanity.com/paper/2406.07855/) (2024)
- [RALL-E: Robust Codec Language Modeling with Chain-of-Thought Prompting for Text-to-Speech Synthesis](https://mlsanity.com/paper/2404.03204/) (2024)
- [A Vector Quantized Approach for Text to Speech Synthesis on Real-World Spontaneous Speech](https://mlsanity.com/paper/2302.04215/) (2023)
- [Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speech](https://mlsanity.com/paper/2106.12896/) (2021)
- [Alignment Knowledge Distillation for Online Streaming Attention-based Speech Recognition](https://mlsanity.com/paper/2103.00422/) (2021)
- [VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attention](https://mlsanity.com/paper/2102.06431/) (2021)
- [More than Words: In-the-Wild Visually-Driven Prosody for Text-to-Speech](https://mlsanity.com/paper/2111.10139/) (2021)
- [Structured State Space Decoder for Speech Recognition and Synthesis](https://mlsanity.com/paper/2210.17098/) (2022)
- [On-device neural speech synthesis](https://mlsanity.com/paper/2109.08710/) (2021)
- [A Multi-Stage Multi-Codebook VQ-VAE Approach to High-Performance Neural TTS](https://mlsanity.com/paper/2209.10887/) (2022)
- [Neural HMMs are all you need (for high-quality attention-free TTS)](https://mlsanity.com/paper/2108.13320/) (2021)
- [OverFlow: Putting flows on top of neural transducers for better TTS](https://mlsanity.com/paper/2211.06892/) (2022)
- [Whispered and Lombard Neural Speech Synthesis](https://mlsanity.com/paper/2101.05313/) (2021)
- [Review of end-to-end speech synthesis technology based on deep learning](https://mlsanity.com/paper/2104.09995/) (2021)
- [A study of latent monotonic attention variants](https://mlsanity.com/paper/2103.16710/) (2021)
- [Exploring Machine Speech Chain for Domain Adaptation and Few-Shot Speaker Adaptation](https://mlsanity.com/paper/2104.03815/) (2021)
- [Combining speakers of multiple languages to improve quality of neural voices](https://mlsanity.com/paper/2108.07737/) (2021)
- [A Virtual Simulation-Pilot Agent for Training of Air Traffic Controllers](https://mlsanity.com/paper/2304.07842/) (2023)
- [Enhancing audio quality for expressive Neural Text-to-Speech](https://mlsanity.com/paper/2108.06270/) (2021)
- [Zero-Shot Long-Form Voice Cloning with Dynamic Convolution Attention](https://mlsanity.com/paper/2201.10375/) (2022)
- [Continuous Autoregressive Modeling with Stochastic Monotonic Alignment for Speech Synthesis](https://mlsanity.com/paper/2502.01084/) (2025)
- [A study on the efficacy of model pre-training in developing neural text-to-speech system](https://mlsanity.com/paper/2110.03857/) (2021)
- [Energy-Based Models For Speech Synthesis](https://mlsanity.com/paper/2310.12765/) (2023)
- [Eliminating stability hallucinations in llm-based tts models via attention guidance](https://mlsanity.com/paper/2509.19852/) (2025)
- [Triple M: A Practical Text-to-speech Synthesis System With Multi-guidance Attention And Multi-band Multi-time LPCNet](https://mlsanity.com/paper/2102.00247/) (2021)
- [Multi-Scale Spectrogram Modelling for Neural Text-to-Speech](https://mlsanity.com/paper/2106.15649/) (2021)
- [PAMA-TTS: Progression-Aware Monotonic Attention for Stable Seq2Seq TTS With Accurate Phoneme Duration Control](https://mlsanity.com/paper/2110.04486/) (2021)
- [AutoTTS: End-to-End Text-to-Speech Synthesis through Differentiable Duration Modeling](https://mlsanity.com/paper/2203.11049/) (2022)

## Abstract (arXiv)

Neural TTS has demonstrated strong capabilities to generate human-like speech with high quality and naturalness, while its generalization to out-of-domain texts is still a challenging task, with regard to the design of attention-based sequence-to-sequence acoustic modeling. Various errors occur in those inputs with unseen context, including attention collapse, skipping, repeating, etc., which limits the broader applications. In this paper, we propose a novel stepwise monotonic attention method in sequence-to-sequence acoustic modeling to improve the robustness on out-of-domain inputs. The method utilizes the strict monotonic property in TTS with constraints on monotonic hard attention that the alignments between inputs and outputs sequence must be not only monotonic but allowing no skipping on inputs. Soft attention could be used to evade mismatch between training and inference. The experimental results show that the proposed method could achieve significant improvements in robustness on out-of-domain scenarios for phoneme-based models, without any regression on the in-domain naturalness test.