mlsanityData to 14 Oct 2026

Revisiting ASR Error Correction with Specialized Models

arXiv 2405.15216 ↗First posted 24 May 2024cs.LG cs.CL cs.SDcited by 5

Not on the map. Why?

Abstract (arXiv)

Language models play a central role in automatic speech recognition (ASR), yet most methods rely on text-only models unaware of ASR error patterns. Recently, large language models (LLMs) have been applied to ASR correction, but introduce latency and hallucination concerns. We revisit ASR error correction with compact seq2seq models, trained on ASR errors from real and synthetic audio. To scale training, we construct synthetic corpora via cascaded TTS and ASR, finding that matching the diversity of realistic error distributions is key. We propose correction-first decoding, where the correction model generates candidates rescored using ASR acoustic scores. With 15x fewer parameters than LLMs, our model achieves 1.5/3.3% WER on LibriSpeech test-clean/other, outperforms LLMs, generalizes across ASR architectures (CTC, Seq2seq, Transducer) and diverse domains, and provides precise corrections in the low-error regime where LLMs struggle.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 45 of its 68 references that are in the map. First posted on arXiv 24 May 2024.

Builds on

45 of 68 references.

References · 45

  1. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2023Mistral 7Bcited by 3,164
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  6. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  7. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  8. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  9. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  10. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  11. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
  12. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 226
33 more
  1. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 187
  2. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 150
  3. 2023On decoder-only architecture for speech-to-text and large language model integrationcited by 145
  4. 2023Prompting Large Language Models with Speech Recognition Abilitiescited by 140
  5. 2020CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordingscited by 132
  6. 2019End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architecturescited by 120
  7. 2017An analysis of incorporating an external language model into a sequence-to-sequence modelcited by 111
  8. 2022Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Languagecited by 110
  9. 2020Hybrid Autoregressive Transducer (hat)cited by 108
  10. 2016Towards better decoding and language model integration in sequence to sequence modelscited by 100
  11. 2020ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Contextcited by 97
  12. 2019QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutionscited by 94
  13. 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 92
  14. 2017Cold Fusion: Training Seq2Seq Models Together with Language Modelscited by 88
  15. 2023Libriheavy: a 50,000 hours ASR corpus with punctuation casing and contextcited by 85
  16. 2020Iterative Pseudo-Labeling for Speech Recognitioncited by 83
  17. 2016Reward Augmented Maximum Likelihood for Neural Structured Predictioncited by 81
  18. 2020Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognitioncited by 80
  19. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 80
  20. 2020A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognitioncited by 73
  21. 2023Generative Speech Recognition Error Correction with Large Language Models and Task-Activating Promptingcited by 72
  22. 2018A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognitioncited by 66
  23. 2019A spelling correction model for end-to-end speech recognitioncited by 63
  24. 2023Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognitioncited by 56
  25. 2019Correction of Automatic Speech Recognition with Transformer Sequence-to-sequence Modelcited by 52
  26. 2023N-best T5: Robust ASR Error Correction using Multiple Input Hypotheses and Constrained Decoding Spacecited by 42
  27. 2024Large Language Models are Efficient Learners of Noise-Robust Speech Recognitioncited by 31
  28. 2021FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognitioncited by 27
  29. 2021Synt++: Utilizing Imperfect Synthetic Data to Improve Speech Recognitioncited by 24
  30. 2021BART based semantic correction for Mandarin automatic speech recognition systemcited by 20
  31. 2022Error Correction in ASR using Sequence-to-Sequence Modelscited by 19
  32. 2018Learning from Past Mistakes: Improving Automatic Speech Recognition Output via Noisy-Clean Phrase Context Modelingcited by 10
  33. 2020ASAPP-ASR: Multistream CNN and Self-Attentive SRU for SOTA Speech Recognitioncited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2025Aligning ASR Evaluation with Human and LLM Judgments: Intelligibility Metrics Using Phonetic, Semantic, and NLI Approachescited by 7
  2. 2024Evolutionary Prompt Design for LLM-Based Post-ASR Error Correctioncited by 4
  3. 2025Listening, Imagining & Refining: A Heuristic Optimized ASR Correction Framework with LLMscited by 1
  4. 2026ReHear: Iterative Pseudo-Label Refinement for Semi-Supervised Speech Recognition via Audio Large Language Modelscited by 1
  5. 2026Diffusion Language Models for Speech Recognitioncited by 0