mlsanityData to 7 Oct 2026

Have best of both worlds: two-pass hybrid and E2E cascading framework for speech recognition

arXiv 2110.04891 ↗First posted 10 Oct 2021cs.CL cs.SD eess.AScited by 7

Not on the map. Why?

Abstract (arXiv)

Hybrid and end-to-end (E2E) systems have their individual advantages, with different error patterns in the speech recognition results. By jointly modeling audio and text, the E2E model performs better in matched scenarios and scales well with a large amount of paired audio-text training data. The modularized hybrid model is easier for customization, and better to make use of a massive amount of unpaired text data. This paper proposes a two-pass hybrid and E2E cascading (HEC) framework to combine the hybrid and E2E model in order to take advantage of both sides, with hybrid in the first pass and E2E in the second pass. We show that the proposed system achieves 8-10% relative word error rate reduction with respect to each individual system. More importantly, compared with the pure E2E system, we show the proposed system has the potential to keep the advantages of hybrid system, e.g., customization and segmentation capabilities. We also show the second pass E2E model in HEC is robust with respect to the change in the first pass hybrid model.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 24 of its 34 references that are in the map. First posted on arXiv 10 Oct 2021.

Builds on

24 of 34 references.

References · 24

  1. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  2. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  3. 2015Listen, Attend and Spellcited by 542
  4. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 232
  5. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
  6. 2017State-of-the-art Speech Recognition With Sequence-to-Sequence Modelscited by 196
  7. 2018Deep context: end-to-end contextual speech recognitioncited by 136
  8. 2020Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Datasetcited by 94
  9. 2020Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognitioncited by 80
  10. 2020A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognitioncited by 73
  11. 2014End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Resultscited by 68
  12. 2020A Better and Faster End-to-End Model for Streaming ASRcited by 68
12 more
  1. 2018A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognitioncited by 66
  2. 2020On the Comparison of Popular End-to-End Models for Large Scale Speech Recognitioncited by 62
  3. 2020Deliberation Model Based Two-Pass End-to-End Speech Recognitioncited by 57
  4. 2020Cascaded encoders for unifying streaming and non-streaming ASRcited by 51
  5. 2016Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognitioncited by 33
  6. 2021Transformer Based Deliberation for Two-Pass Speech Recognitioncited by 29
  7. 2021Reducing Streaming ASR Model Delay with Self Alignmentcited by 16
  8. 2018Advancing Acoustic-to-Word CTC Modelcited by 15
  9. 2021A Light-weight contextual spelling correction model for customizing transducer-based speech recognition systemscited by 11
  10. 2019Integrating Source-channel and Attention-based Sequence-to-sequence Models for Speech Recognitioncited by 8
  11. 2020High-Accuracy and Low-Latency Speech Recognition with Two-Head Contextual Layer Trajectory LSTM Modelcited by 8
  12. 2021Combining Frame-Synchronous and Label-Synchronous Systems for Speech Recognitioncited by 3

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 2; 2023: 2; 2024: 1.

7 papers.

  1. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  2. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  3. 2022Towards Contextual Spelling Correction for Customization of End-to-end Speech Recognition Systemscited by 17
  4. 2022SoftCorrect: Error Correction with Soft Detection for Automatic Speech Recognitioncited by 9
  5. 2023Improving Contextual Spelling Correction by External Acoustics Attention and Semantic Aware Data Augmentationcited by 9
  6. 2024Joint Beam Search Integrating CTC, Attention, and Transducer Decoderscited by 4
  7. 2023Improving CTC-AED model with integrated-CTC and auxiliary loss regularizationcited by 0