# Forward Attention in Sequence-to-sequence Acoustic Modelling for Speech Synthesis

arXiv: https://arxiv.org/abs/1807.06736 · HTML: https://mlsanity.com/paper/1807.06736/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2018.

## Built on by (most cited first, 17 of 17)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Features](https://mlsanity.com/paper/2203.03191/) (2022)
- [VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attention](https://mlsanity.com/paper/2102.06431/) (2021)
- [NatiQ: An End-to-end Text-to-Speech System for Arabic](https://mlsanity.com/paper/2206.07373/) (2022)
- [Review of end-to-end speech synthesis technology based on deep learning](https://mlsanity.com/paper/2104.09995/) (2021)
- [A study of latent monotonic attention variants](https://mlsanity.com/paper/2103.16710/) (2021)
- [Singing-Tacotron: Global duration control attention and dynamic filter for End-to-end singing voice synthesis](https://mlsanity.com/paper/2202.07907/) (2022)
- [Robust and Unbounded Length Generalization in Autoregressive Transformer-Based Text-to-Speech](https://mlsanity.com/paper/2410.22179/) (2024)
- [Enhancing audio quality for expressive Neural Text-to-Speech](https://mlsanity.com/paper/2108.06270/) (2021)
- [Zero-Shot Long-Form Voice Cloning with Dynamic Convolution Attention](https://mlsanity.com/paper/2201.10375/) (2022)
- [Investigation of Japanese PnG BERT language model in text-to-speech synthesis for pitch accent language](https://mlsanity.com/paper/2212.08321/) (2022)
- [Eliminating stability hallucinations in llm-based tts models via attention guidance](https://mlsanity.com/paper/2509.19852/) (2025)
- [Triple M: A Practical Text-to-speech Synthesis System With Multi-guidance Attention And Multi-band Multi-time LPCNet](https://mlsanity.com/paper/2102.00247/) (2021)
- [Nana-HDR: A Non-attentive Non-autoregressive Hybrid Model for TTS](https://mlsanity.com/paper/2109.13673/) (2021)
- [PAMA-TTS: Progression-Aware Monotonic Attention for Stable Seq2Seq TTS With Accurate Phoneme Duration Control](https://mlsanity.com/paper/2110.04486/) (2021)
- [Singing Voice Synthesis Based on a Musical Note Position-Aware Attention Mechanism](https://mlsanity.com/paper/2212.13703/) (2022)
- [Rhythm-controllable Attention with High Robustness for Long Sentence Speech Synthesis](https://mlsanity.com/paper/2306.02593/) (2023)

## Abstract (arXiv)

This paper proposes a forward attention method for the sequenceto- sequence acoustic modeling of speech synthesis. This method is motivated by the nature of the monotonic alignment from phone sequences to acoustic sequences. Only the alignment paths that satisfy the monotonic condition are taken into consideration at each decoder timestep. The modified attention probabilities at each timestep are computed recursively using a forward algorithm. A transition agent for forward attention is further proposed, which helps the attention mechanism to make decisions whether to move forward or stay at each decoder timestep. Experimental results show that the proposed forward attention method achieves faster convergence speed and higher stability than the baseline attention method. Besides, the method of forward attention with transition agent can also help improve the naturalness of synthetic speech and control the speed of synthetic speech effectively.