mlsanityData to 7 Oct 2026

Effect and Analysis of Large-scale Language Model Rescoring on Competitive ASR Systems

arXiv 2204.00212 ↗First posted 1 Apr 2022cs.CL cs.SD eess.AScited by 23

Not on the map. Why?

Abstract (arXiv)

Large-scale language models (LLMs) such as GPT-2, BERT and RoBERTa have been successfully applied to ASR N-best rescoring. However, whether or how they can benefit competitive, near state-of-the-art ASR systems remains unexplored. In this study, we incorporate LLM rescoring into one of the most competitive ASR baselines: the Conformer-Transducer model. We demonstrate that consistent improvement is achieved by the LLM's bidirectionality, pretraining, in-domain finetuning and context augmentation. Furthermore, our lexical analysis sheds light on how each of these components may be contributing to the ASR performance.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 20 of its 34 references that are in the map. First posted on arXiv 1 Apr 2022.

Builds on

20 of 34 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019Language Models as Knowledge Bases?cited by 1,922
  5. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  6. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  7. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  8. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 553
  9. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  10. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 184
  11. 2021Advancing RNN Transducer Technology for Speech Recognitioncited by 44
  12. 2017English Conversational Telephone Speech Recognition by Humans and Machinescited by 40
8 more
  1. 2020Distilling the Knowledge of BERT for Sequence-to-Sequence ASRcited by 38
  2. 2020Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboardcited by 36
  3. 2022RescoreBERT: Discriminative Speech Recognition Rescoring with BERTcited by 34
  4. 2015Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognitioncited by 26
  5. 2021Innovative Bert-based Reranking Language Models for Speech Recognitioncited by 23
  6. 2022Knowledge Transfer from Large-scale Pretrained Language Models to End-to-end Speech Recognizerscited by 19
  7. 2021Adapting GPT, GPT-2 and BERT Language Models for Speech Recognitioncited by 16
  8. 2021ASR Rescoring and Confidence Estimation with ELECTRAcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 10; 2024: 5; 2025: 4; 2026: 2.

23 papers.

Sort:

By area:

  1. 2023HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Modelscited by 41
  2. 2023Multi-stage Large Language Model Correction for Speech Recognitioncited by 13
  3. 2022BERT Meets CTC: New Formulation of End-to-End Speech Recognition with Pre-trained Masked Language Modelcited by 11
  4. 2025When Large Language Models Meet Speech: A Survey on Integration Approachescited by 9
  5. 2022BECTRA: Transducer-based End-to-End ASR with BERT-Enhanced Encodercited by 8
  6. 2024SALSA: Speedy ASR-LLM Synchronous Aggregationcited by 8
  7. 2023Exploring the Integration of Large Language Models into Automatic Speech Recognition Systems: An Empirical Studycited by 7
  8. 2023Contextual Biasing of Named-Entities with Large Language Modelscited by 7
  9. 2023Understanding Shared Speech-Text Representationscited by 6
  10. 2024ProGRes: Prompted Generative Rescoring on ASR n-Bestcited by 5
  11. 2025Delayed Fusion: Integrating Large Language Models into First-Pass Decoding in End-to-end Speech Recognitioncited by 5
  12. 2024Towards interfacing large language models with ASR systems using confidence measures and promptingcited by 4
11 more
  1. 2024Robust ASR Error Correction with Conservative Data Filteringcited by 3
  2. 2025Drax: Speech Recognition with Discrete Flow Matchingcited by 3
  3. 2023Masked Audio Text Encoders are Effective Multi-Modal Rescorerscited by 2
  4. 2023Multiple Representation Transfer from Large Language Models to End-to-End ASR Systemscited by 2
  5. 2023Harnessing the Zero-Shot Power of Instruction-Tuned Large Language Model in End-to-End Speech Recognitioncited by 2
  6. 2023Correction Focused Language Model Training for Speech Recognitioncited by 2
  7. 2023On the N-gram Approximation of Pre-trained Language Modelscited by 1
  8. 2024Speech Recognition Rescoring with Large Speech-Text Foundation Modelscited by 1
  9. 2025MTLM: Incorporating Bidirectional Text Information to Enhance Language Model Training in Speech Recognition Systemscited by 0
  10. 2026Rethinking Human-Aligned Evaluation: An Analysis of Semantic Metrics Beyond WERcited by 0
  11. 2026Jev Matches 7B Language Models for Speech-Neuroprosthesis Rescoringcited by 0