mlsanityData to 7 Oct 2026

Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition

arXiv 2011.01991 ↗First posted 3 Nov 2020eess.AS cs.CL cs.LGcited by 80

Not on the map. Why?

Abstract (arXiv)

The external language models (LM) integration remains a challenging task for end-to-end (E2E) automatic speech recognition (ASR) which has no clear division between acoustic and language models. In this work, we propose an internal LM estimation (ILME) method to facilitate a more effective integration of the external LM with all pre-existing E2E models with no additional model training, including the most popular recurrent neural network transducer (RNN-T) and attention-based encoder-decoder (AED) models. Trained with audio-transcript pairs, an E2E model implicitly learns an internal LM that characterizes the training data in the source domain. With ILME, the internal LM scores of an E2E model are estimated and subtracted from the log-linear interpolation between the scores of the E2E model and the external LM. The internal LM scores are approximated as the output of an E2E model when eliminating its acoustic components. ILME can alleviate the domain mismatch between training and testing, or improve the multi-domain E2E ASR. Experimented with 30K-hour trained RNN-T and AED models, ILME achieves up to 15.5% and 6.8% relative word error rate reductions from Shallow Fusion on out-of-domain LibriSpeech and in-domain Microsoft production test sets, respectively.

Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 18; 2023: 25; 2024: 11; 2025: 7; 2026: 3.

80 papers.

Sort:

By area:

  1. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  2. 2023End-to-End Speech Recognition: A Surveycited by 127
  3. 2021Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusioncited by 100
  4. 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 90
  5. 2024LLM Internal States Reveal Hallucination Risk Faced With a Querycited by 80
  6. 2023Can Generative Large Language Models Perform ASR Error Correction?cited by 56
  7. 2023Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognitioncited by 48
  8. 2021Semantic Distance: A New Metric for ASR Performance Analysis Towards Spoken Language Understandingcited by 40
  9. 2021Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognitioncited by 37
  10. 2021SynthASR: Unlocking Synthetic Data for Speech Recognitioncited by 34
  11. 2021Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognitioncited by 32
  12. 2021Factorized Neural Transducer for Efficient Language Model Adaptationcited by 31
38 more
  1. 2021Librispeech Transducer Model with Internal Language Model Prior Correctioncited by 26
  2. 2021Investigating Methods to Improve Language Model Integration for Attention-based Encoder-Decoder ASR Modelscited by 25
  3. 2024ASR Error Correction using Large Language Modelscited by 23
  4. 2023Can Contextual Biasing Remain Effective with Whisper and GPT-2?cited by 21
  5. 2022Integrating Text Inputs For Training and Adapting RNN Transducer ASR Modelscited by 20
  6. 2021Fast Text-Only Domain Adaptation of RNN-Transducer Prediction Networkcited by 19
  7. 2021Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognitioncited by 19
  8. 2021Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognitioncited by 17
  9. 2021On Language Model Integration for RNN Transducer based Speech Recognitioncited by 17
  10. 2022Modular Hybrid Autoregressive Transducercited by 17
  11. 2022Efficient Training of Neural Transducer for Speech Recognitioncited by 14
  12. 2022Improving Rare Word Recognition with LM-aware MWER Trainingcited by 12
  13. 2022Minimising Biasing Word Errors for Contextual ASR with the Tree-Constrained Pointer Generatorcited by 11
  14. 2022Fast and accurate factorized neural transducer for text adaption of end-to-end speech recognition modelscited by 11
  15. 2023Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical Distillationcited by 11
  16. 2023CB-Conformer: Contextual biasing Conformer for biased word recognitioncited by 11
  17. 2024Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognitioncited by 10
  18. 2022Internal Language Model Estimation Through Explicit Context Vector Learning for Attention-based Encoder-decoder ASRcited by 9
  19. 2023JEIT: Joint End-to-End Model and Internal Language Model Training for Speech Recognitioncited by 9
  20. 2021Comparing the Benefit of Synthetic Training Data for Various Automatic Speech Recognition Architecturescited by 8
  21. 2023Approximate Nearest Neighbour Phrase Mining for Contextual Speech Recognitioncited by 8
  22. 2024SALSA: Speedy ASR-LLM Synchronous Aggregationcited by 8
  23. 2021Have best of both worlds: two-pass hybrid and E2E cascading framework for speech recognitioncited by 7
  24. 2022A Simple Baseline for Domain Adaptation in End to End ASR Systems Using Synthetic Datacited by 7
  25. 2025NGPU-LM: GPU-Accelerated N-Gram Language Model for Context-Biasing in Greedy ASR Decodingcited by 7
  26. 2023Graph Neural Networks for Contextual ASR with the Tree-Constrained Pointer Generatorcited by 6
  27. 2023Acoustic Model Fusion for End-to-end Speech Recognitioncited by 6
  28. 2023Personalization of CTC-based End-to-End Speech Recognition Using Pronunciation-Driven Subword Tokenizationcited by 6
  29. 2022A Likelihood Ratio based Domain Adaptation Method for E2E Modelscited by 5
  30. 2022Adaptive Discounting of Implicit Language Models in RNN-Transducerscited by 5
  31. 2022Internal Language Model Estimation based Language Model Fusion for Cross-Domain Code-Switching Speech Recognitioncited by 5
  32. 2022Optimizing Bilingual Neural Transducer with Synthetic Code-switching Text Generationcited by 5
  33. 2022Joint Audio/Text Training for Transformer Rescorer of Streaming Speech Recognitioncited by 5
  34. 2023Text-only Domain Adaptation using Unified Speech-Text Representation in Transducercited by 5
  35. 2024Revisiting ASR Error Correction with Specialized Modelscited by 5
  36. 2025Delayed Fusion: Integrating Large Language Models into First-Pass Decoding in End-to-end Speech Recognitioncited by 5
  37. 2025Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translationcited by 5
  38. 2023Improving RNN-Transducers with Acoustic LookAheadcited by 4

50 most cited of 80 All 80 (JSON)