Contextual-Utterance Training for Automatic Speech Recognition
Not on the map. Why?
Abstract (arXiv)
Recent studies of streaming automatic speech recognition (ASR) recurrent neural network transducer (RNN-T)-based systems have fed the encoder with past contextual information in order to improve its word error rate (WER) performance. In this paper, we first propose a contextual-utterance training technique which makes use of the previous and future contextual utterances in order to do an implicit adaptation to the speaker, topic and acoustic environment. Also, we propose a dual-mode contextual-utterance training technique for streaming automatic speech recognition (ASR) systems. This proposed approach allows to make a better use of the available acoustic context in streaming models by distilling "in-place" the knowledge of a teacher, which is able to see both past and future contextual utterances, to the student which can only see the current and past contextual utterances. The experimental results show that a conformer-transducer system trained with the proposed techniques outperforms the same system trained with the classical RNN-T loss. Specifically, the proposed technique is able to reduce both the WER and the average last token emission latency by more than 6% and 40ms relative, respectively.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 17 of its 21 references that are in the map. First posted on arXiv 27 Oct 2022.
Builds on
17 of 21 references.
References · 17
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
- 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
- 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 235
- 2020Contextual RNN-T For Open Domain ASRcited by 85
- 2019Improving RNN Transducer Modeling for End-to-End Speech Recognitioncited by 68
- 2019SpecAugment on Large Scale Datasetscited by 60
- 2021Context-Aware Transformer Transducer for Speech Recognitioncited by 56
- 2020Efficient Knowledge Distillation for RNN-Transducer Modelscited by 38
- 2021Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformerscited by 25
- 2021CoDERT: Distilling Encoder Representations with Co-learning for Transducer-based Speech Recognitioncited by 11
- 2022Improving the fusion of acoustic and text representations in RNN-Tcited by 11
5 more
- 2020Improving RNN-T ASR Accuracy Using Context Audiocited by 8
- 2020Speaker-aware speech-transformercited by 7
- 2021Collaborative Training of Acoustic Encoders for Speech Recognitioncited by 7
- 2017Accelerating recurrent neural network training using sequence bucketing and multi-GPU data parallelizationcited by 4
- 2017Direct Acoustics-to-Word Models for English Conversational Speech Recognitioncited by 3
Built on by
Citing papers per year: 2024: 1.
1 paper.
- 2024Promptformer: Prompted Conformer Transducer for ASRcited by 3
- 2024Promptformer: Prompted Conformer Transducer for ASR14 Jan 2024
Similar, not cited
Close in wording, no citation either way
- 2021Bridging the gap between streaming and non-streaming ASR systems bydistilling ensembles of CTC and RNN-T models
- 2026SENS-ASR: Semantic Embedding injection in Neural-transducer for Streaming Automatic Speech Recognition
- 2020Exploring Pre-training with Alignments for RNN Transducer based End-to-End Speech Recognition
- 2026Speak in Context: Multilingual ASR with Speech Context Alignment via Contrastive Learning
- 2024Fast Streaming Transducer ASR Prototyping via Knowledge Distillation with Whisper
- 2021Knowledge Distillation for Neural Transducers from Large Self-Supervised Pre-trained Models
- 2021Hierarchical Transformer-based Large-Context End-to-end ASR with Large-Context Knowledge Distillation
- 2023Knowledge Distillation from Non-streaming to Streaming ASR Encoder using Auxiliary Non-streaming Layer