Weak Alignment Supervision from Hybrid Model Improves End-to-end ASR
Not on the map. Why?
Abstract (arXiv)
In this paper, we aim to create weak alignment supervision from an existing hybrid system to aid the end-to-end modeling of automatic speech recognition. Towards this end, we use the existing hybrid ASR system to produce triphone alignments of the training audios. We then create a cross-entropy loss at a certain layer of the encoder using the derived alignments. In contrast to the general one-hot cross-entropy losses, here we use a cross-entropy loss with a label smoothing parameter to regularize the supervision. As a comparison, we also conduct the experiments with one-hot cross-entropy losses and CTC losses with loss weighting. The results show that placing the weak alignment supervision with the label smoothing parameter of 0.5 at the third encoder layer outperforms the other two approaches and leads to about 5% relative WER reduction on the TED-LIUM 2 dataset over the baseline. We see similar improvements when applying the method out-of-the-box on a Tagalog end-to-end ASR system.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 30 of its 47 references that are in the map. First posted on arXiv 24 Nov 2023.
Builds on
30 of 47 references.
References · 30
- 2017Attention Is All You Needcited by 59,932
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
- 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
- 2017A Survey on Multi-Task Learningcited by 838
- 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
- 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
- 2015Listen, Attend and Spellcited by 542
- 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 374
18 more
- 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 254
- 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
- 2015End-to-End Attention-based Large Vocabulary Speech Recognitioncited by 148
- 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
- 2023End-to-End Speech Recognition: A Surveycited by 127
- 2018Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducercited by 95
- 2021Intermediate Loss Regularization for CTC-based Speech Recognitioncited by 95
- 2015Multi-task Sequence to Sequence Learningcited by 94
- 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 90
- 2021Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for Automatic Speech Recognitioncited by 33
- 2023Improving Massively Multilingual ASR With Auxiliary CTC Objectivescited by 31
- 2018RETURNN as a Generic Flexible Neural Toolkit with Application to Translation and Speech Recognitioncited by 30
- 2017Multitask Learning with Low-Level Auxiliary Tasks for Encoder-Decoder Based Speech Recognitioncited by 29
- 2020Improving RNN Transducer Based ASR with Auxiliary Taskscited by 27
- 2019RNN-T For Latency Controlled ASR With Improved Beam Searchcited by 19
- 2022A Study of Transducer based End-to-End ASR with ESPnet: Architecture, Auxiliary Loss and Decoding Strategiescited by 15
- 2021Self-Supervised Speaker Verification with Simple Siamese Network and Self-Supervised Regularizationcited by 14
- 2023Chunked Attention-based Encoder-Decoder Model for Streaming Speech Recognitioncited by 7
Built on by
Citing papers per year: 2024: 1.
1 paper.