A Deliberation-based Joint Acoustic and Text Decoder
Not on the map. Why?
Abstract (arXiv)
We propose a new two-pass E2E speech recognition model that improves ASR performance by training on a combination of paired data and unpaired text data. Previously, the joint acoustic and text decoder (JATD) has shown promising results through the use of text data during model training and the recently introduced deliberation architecture has reduced recognition errors by leveraging first-pass decoding results. Our method, dubbed Deliberation-JATD, combines the spelling correcting abilities of deliberation with JATD's use of unpaired text data to further improve performance. The proposed model produces substantial gains across multiple test sets, especially those focused on rare words, where it reduces word error rate (WER) by between 12% and 22.5% relative. This is done without increasing model size or requiring multi-stage training, making Deliberation-JATD an efficient candidate for on-device applications.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 32 references that are in the map. First posted on arXiv 23 Mar 2023.
Builds on
22 of 32 references.
References · 22
- 2017Attention Is All You Needcited by 59,932
- 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
- 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 660
- 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 254
- 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
- 2017State-of-the-art Speech Recognition With Sequence-to-Sequence Modelscited by 196
- 2017An analysis of incorporating an external language model into a sequence-to-sequence modelcited by 109
- 2016Towards better decoding and language model integration in sequence to sequence modelscited by 100
- 2018Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducercited by 95
- 2017Cold Fusion: Training Seq2Seq Models Together with Language Modelscited by 87
- 2019Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modelingcited by 86
- 2019Recognizing long-form speech using streaming end-to-end modelscited by 77
10 more
- 2019Two-Pass End-to-End Speech Recognitioncited by 62
- 2020Deliberation Model Based Two-Pass End-to-End Speech Recognitioncited by 57
- 2019Almost Unsupervised Text to Speech and Automatic Speech Recognitioncited by 41
- 2018Cycle-consistency training for end-to-end speech recognitioncited by 31
- 2021Transformer Based Deliberation for Two-Pass Speech Recognitioncited by 29
- 2020Improving Tail Performance of a Deliberation E2E ASR Model Using a Large Text Corpuscited by 24
- 2020Multitask Training with Text Data for End-to-End Speech Recognitioncited by 19
- 2019Improving Performance of End-to-End ASR on Numeric Sequencescited by 17
- 2019Learn Spelling from Teachers: Transferring Knowledge from Language Models to Sequence-to-Sequence Speech Recognitioncited by 12
- 2018Transfer learning of language-independent end-to-end ASR with language model fusioncited by 10
Built on by
Citing papers per year: 2023: 1.
1 paper.