mlsanityData to 7 Oct 2026

Two-Pass End-to-End Speech Recognition

arXiv 1908.10992 ↗First posted 29 Aug 2019cs.CL cs.SD eess.AScited by 62

Not on the map. Why?

Abstract (arXiv)

The requirements for many applications of state-of-the-art speech recognition systems include not only low word error rate (WER) but also low latency. Specifically, for many use-cases, the system must be able to decode utterances in a streaming fashion and faster than real-time. Recently, a streaming recurrent neural network transducer (RNN-T) end-to-end (E2E) model has shown to be a good candidate for on-device speech recognition, with improved WER and latency metrics compared to conventional on-device models [1]. However, this model still lags behind a large state-of-the-art conventional model in quality [2]. On the other hand, a non-streaming E2E Listen, Attend and Spell (LAS) model has shown comparable quality to large conventional models [3]. This work aims to bring the quality of an E2E streaming model closer to that of a conventional system by incorporating a LAS network as a second-pass component, while still abiding by latency constraints. Our proposed two-pass model achieves a 17%-22% relative reduction in WER compared to RNN-T alone and increases latency by a small fraction over RNN-T.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 21; 2023: 14; 2024: 10; 2025: 2; 2026: 1.

62 papers.

Sort:

By area:

  1. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  2. 2023End-to-End Speech Recognition: A Surveycited by 127
  3. 2022UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Unitscited by 60
  4. 2022RescoreBERT: Discriminative Speech Recognition Rescoring with BERTcited by 34
  5. 2021Instant One-Shot Word-Learning for Context-Specific Neural Sequence-to-Sequence Speech Recognitioncited by 30
  6. 2021Transformer Based Deliberation for Two-Pass Speech Recognitioncited by 29
  7. 2022CTC Alignments Improve Autoregressive Translationcited by 25
  8. 2021Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Taskscited by 24
  9. 2024It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognitioncited by 22
  10. 2023Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Taskscited by 20
  11. 2024Contextualized Automatic Speech Recognition with Attention-Based Bias Phrase Boosted Beam Searchcited by 19
  12. 2022A Study on the Integration of Pre-trained SSL, ASR, LM and SLU Models for Spoken Language Understandingcited by 18
38 more
  1. 2022E2E Segmenter: Joint Segmenting and Decoding for Long-Form ASRcited by 17
  2. 2022Modular Hybrid Autoregressive Transducercited by 17
  3. 2023Low-rank Adaptation of Large Language Model Rescoring for Parameter-Efficient Speech Recognitioncited by 13
  4. 2022Conformer Based Elderly Speech Recognition System for Alzheimer's Disease Detectioncited by 12
  5. 2024Decoder-only Architecture for Streaming End-to-end Speech Recognitioncited by 12
  6. 2021FSR: Accelerating the Inference Process of Transducer-Based Models by Applying Fast-Skip Regularizationcited by 11
  7. 2022BERT Meets CTC: New Formulation of End-to-End Speech Recognition with Pre-trained Masked Language Modelcited by 11
  8. 2022Improving Deliberation by Text-Only and Semi-Supervised Trainingcited by 10
  9. 2021Residual Energy-Based Models for End-to-End Speech Recognitioncited by 9
  10. 2021Listen with Intent: Improving Speech Recognition with Audio-to-Intent Front-Endcited by 8
  11. 2022Streaming parallel transducer beam search with fast-slow cascaded encoderscited by 8
  12. 2022Two-Pass Low Latency End-to-End Spoken Language Understandingcited by 8
  13. 2022BECTRA: Transducer-based End-to-End ASR with BERT-Enhanced Encodercited by 8
  14. 2021TSNAT: Two-Step Non-Autoregressvie Transformer Models for Speech Recognitioncited by 7
  15. 2021SimulLR: Simultaneous Lip Reading Transducer with Attention-Guided Adaptive Memorycited by 7
  16. 20224D ASR: Joint modeling of CTC, Attention, Transducer, and Mask-Predict decoderscited by 7
  17. 2023Exploring the Integration of Large Language Models into Automatic Speech Recognition Systems: An Empirical Studycited by 7
  18. 2021Lattention: Lattice-attention in ASR rescoringcited by 6
  19. 2022Improving Mandarin Speech Recogntion with Block-augmented Transformercited by 6
  20. 2022Multi-stage Progressive Compression of Conformer Transducer for On-device Speech Recognitioncited by 6
  21. 2022Scaling Up Deliberation for Multilingual ASRcited by 6
  22. 2021A study of latent monotonic attention variantscited by 5
  23. 2022Joint Audio/Text Training for Transformer Rescorer of Streaming Speech Recognitioncited by 5
  24. 2023TAPIR: Learning Adaptive Revision for Incremental Natural Language Understanding with a Two-Pass Modelcited by 5
  25. 2022Two-pass Decoding and Cross-adaptation Based System Combination of End-to-end Conformer and Hybrid TDNN ASR Systemscited by 4
  26. 2022Monotonic segmental attention for automatic speech recognitioncited by 4
  27. 2024Let's Fuse Step by Step: A Generative Fusion Decoding Algorithm with LLMs for Robust and Instruction-Aware ASR and OCRcited by 4
  28. 2024Joint Beam Search Integrating CTC, Attention, and Transducer Decoderscited by 4
  29. 2024Benchmarking Japanese Speech Recognition on ASR-LLM Setups with Multi-Pass Augmented Generative Error Correctioncited by 4
  30. 2021Bridging the gap between streaming and non-streaming ASR systems bydistilling ensembles of CTC and RNN-T modelscited by 3
  31. 2022E2E Segmentation in a Two-Pass Cascaded Encoder ASR Modelcited by 3
  32. 2024Transformer-based Model for ASR N-Best Rescoring and Rewritingcited by 3
  33. 2024Speculative Speech Recognition by Audio-Prefixed Low-Rank Adaptation of Language Modelscited by 3
  34. 2021Raw Waveform Encoder with Multi-Scale Globally Attentive Locally Recurrent Networks for End-to-End Speech Recognitioncited by 2
  35. 2021Towards efficient end-to-end speech recognition with biologically-inspired neural networkscited by 2
  36. 2023Masked Audio Text Encoders are Effective Multi-Modal Rescorerscited by 2
  37. 2023Modality Confidence Aware Training for Robust End-to-End Spoken Language Understandingcited by 2
  38. 2023Harnessing the Zero-Shot Power of Instruction-Tuned Large Language Model in End-to-End Speech Recognitioncited by 2

50 most cited of 62 All 62 (JSON)