Supervised Attention in Sequence-to-Sequence Models for Speech Recognition
Not on the map. Why?
Abstract (arXiv)
Attention mechanism in sequence-to-sequence models is designed to model the alignments between acoustic features and output tokens in speech recognition. However, attention weights produced by models trained end to end do not always correspond well with actual alignments, and several studies have further argued that attention weights might not even correspond well with the relevance attribution of frames. Regardless, visual similarity between attention weights and alignments is widely used during training as an indicator of the models quality. In this paper, we treat the correspondence between attention weights and alignments as a learning problem by imposing a supervised attention loss. Experiments have shown significant improved performance, suggesting that learning the alignments well during training critically determines the performance of sequence-to-sequence models.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 27 of its 38 references that are in the map. First posted on arXiv 25 Apr 2022.
Builds on
27 of 38 references.
References · 27
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
- 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
- 2019Attention is not Explanationcited by 916
- 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
- 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
- 2019Attention is not not Explanationcited by 629
- 2015Listen, Attend and Spellcited by 542
- 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
- 2015Attention-Based Models for Speech Recognitioncited by 400
- 2019Is Attention Interpretable?cited by 387
- 2016Image Captioning with Semantic Attentioncited by 260
- 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 254
15 more
- 2016Professor Forcing: A New Algorithm for Training Recurrent Networkscited by 214
- 2015End-to-End Attention-based Large Vocabulary Speech Recognitioncited by 148
- 2017Monotonic Chunkwise Attentioncited by 82
- 2015EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decodingcited by 73
- 2014End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Resultscited by 68
- 2016Rationale-Augmented Convolutional Neural Networks for Text Classificationcited by 39
- 2016Neural Machine Translation with Supervised Attentioncited by 31
- 2020Semi-supervised ASR by End-to-end Self-trainingcited by 29
- 2019Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutionscited by 27
- 2016Supervised Attentions for Neural Machine Translationcited by 8
- 2020Differentiable Window for Dynamic Local Attentioncited by 8
- 2021On the Difficulty of Segmenting Words with Attentioncited by 6
- 2018Unsupervised Word Segmentation from Speech with Attentioncited by 4
- 2019Empirical Evaluation of Sequence-to-Sequence Models for Word Discovery in Low-resource Settingscited by 3
- 2021Attention Forcing for Machine Translationcited by 3
Built on by
Citing papers per year: 2024: 1; 2025: 2.
3 papers.