mlsanityData to 14 Oct 2026

A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognition

arXiv 1807.10857 ↗First posted 27 Jul 2018eess.AS cs.AI cs.CLcited by 66

Not on the map. Why?

Abstract (arXiv)

Attention-based recurrent neural encoder-decoder models present an elegant solution to the automatic speech recognition problem. This approach folds the acoustic model, pronunciation model, and language model into a single network and requires only a parallel corpus of speech and text for training. However, unlike in conventional approaches that combine separate acoustic and language models, it is not clear how to use additional (unpaired) text. While there has been previous work on methods addressing this problem, a thorough comparison among methods is still lacking. In this paper, we compare a suite of past methods and some of our own proposed methods for using unpaired text data to improve encoder-decoder models. For evaluation, we use the medium-sized Switchboard data set and the large-scale Google voice search and dictation data sets. Our results confirm the benefits of using unpaired text across a range of methods and data sets. Surprisingly, for first-pass decoding, the rather simple approach of shallow fusion performs best across data sets. However, for Google data sets we find that cold fusion has a lower oracle error rate and outperforms other approaches after second-pass rescoring on the Google voice search data set.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). First posted on arXiv 27 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 23; 2022: 9; 2023: 18; 2024: 5; 2025: 7; 2026: 4.

66 papers.

Sort:

By area:

  1. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 406
  2. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 140
  3. 2021Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusioncited by 101
  4. 2021Context-Aware Transformer Transducer for Speech Recognitioncited by 56
  5. 2023HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Modelscited by 42
  6. 2021Personalization Strategies for End-to-End Speech Recognition Systemscited by 26
  7. 2021Investigating Methods to Improve Language Model Integration for Attention-based Encoder-Decoder ASR Modelscited by 25
  8. 2023An Overview on Language Models: Recent Developments and Outlookcited by 22
  9. 2022Knowledge Transfer from Large-scale Pretrained Language Models to End-to-end Speech Recognizerscited by 19
  10. 2021Efficiently Fusing Pretrained Acoustic and Linguistic Encoders for Low-resource Speech Recognitioncited by 17
  11. 2021Transformer Language Models with LSTM-based Cross-utterance Information Representationcited by 15
  12. 2023Fine-tuning Strategies for Faster Inference using Speech Self-Supervised Models: A Comparative Studycited by 15
38 more
  1. 2023Efficient Domain Adaptation for Speech Foundation Modelscited by 13
  2. 2023Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical Distillationcited by 11
  3. 2021WNARS: WFST based Non-autoregressive Streaming End-to-End Speech Recognitioncited by 10
  4. 2024Failing Forward: Improving Generative Error Correction for ASR with Synthetic Data and Retrieval Augmentationcited by 10
  5. 2021Lookup-Table Recurrent Language Models for Long Tail Speech Recognitioncited by 9
  6. 2022Internal Language Model Estimation Through Explicit Context Vector Learning for Attention-based Encoder-decoder ASRcited by 9
  7. 2021Language model fusion for streaming end to end speech recognitioncited by 8
  8. 2021Have best of both worlds: two-pass hybrid and E2E cascading framework for speech recognitioncited by 7
  9. 2022Improving Mandarin End-to-End Speech Recognition with Word N-gram Language Modelcited by 6
  10. 2022Make More of Your Data: Minimal Effort Data Augmentation for Automatic Speech Recognition and Translationcited by 6
  11. 2023Acoustic Model Fusion for End-to-end Speech Recognitioncited by 6
  12. 2021On-Device Personalization of Automatic Speech Recognition Models for Disordered Speechcited by 5
  13. 2022Joint Audio/Text Training for Transformer Rescorer of Streaming Speech Recognitioncited by 5
  14. 2023Adding guardrails to advanced chatbotscited by 5
  15. 2024Revisiting ASR Error Correction with Specialized Modelscited by 5
  16. 2025Delayed Fusion: Integrating Large Language Models into First-Pass Decoding in End-to-end Speech Recognitioncited by 5
  17. 2025GEC-RAG: Improving Generative Error Correction via Retrieval-Augmented Generation for Automatic Speech Recognition Systemscited by 5
  18. 2021Fine-tuning of Pre-trained End-to-end Speech Recognition with Generative Adversarial Networkscited by 4
  19. 2021Relaxed Attention: A Simple Method to Boost Performance of End-to-End Automatic Speech Recognitioncited by 4
  20. 2021Integrating Knowledge in End-to-End Automatic Speech Recognition for Mandarin-English Code-Switchingcited by 4
  21. 2024Towards interfacing large language models with ASR systems using confidence measures and promptingcited by 4
  22. 2025LegoSLM: Connecting LLM with Speech Encoder using CTC Posteriorscited by 4
  23. 2021Transformer-based ASR Incorporating Time-reduction Layer and Fine-tuning with Self-Knowledge Distillationcited by 3
  24. 2021Overcoming Domain Mismatch in Low Resource Sequence-to-Sequence ASR Models using Hybrid Generated Pseudotranscriptscited by 3
  25. 2023DistriBlock: Identifying adversarial audio samples by leveraging characteristics of the output distributioncited by 3
  26. 2021Semi-supervised transfer learning for language expansion of end-to-end speech recognition models to low-resource languagescited by 2
  27. 2022Internal Language Model Estimation based Adaptive Language Model Fusion for Domain Adaptationcited by 2
  28. 2023External Language Model Integration for Factorized Neural Transducerscited by 2
  29. 2023Multiple Representation Transfer from Large Language Models to End-to-End ASR Systemscited by 2
  30. 2023Augmenting text for spoken language understanding with Large Language Modelscited by 2
  31. 2023Improved Factorized Neural Transducer Model For text-only Domain Adaptationcited by 2
  32. 2021LT-LM: a novel non-autoregressive language model for single-shot lattice rescoringcited by 1
  33. 2021Towards Continual Entity Learning in Language Models for Conversational Agentscited by 1
  34. 2022End-to-end speech recognition modeling from de-identified datacited by 1
  35. 2022Fusing Sentence Embeddings Into LSTM-based Autoregressive Language Modelscited by 1
  36. 2022Memory Augmented Lookup Dictionary based Language Modeling for Automatic Speech Recognitioncited by 1
  37. 2023Mask The Bias: Improving Domain-Adaptive Generalization of CTC-based ASR with Internal Language Model Estimationcited by 1
  38. 2023VILAS: Exploring the Effects of Vision and Language Context in Automatic Speech Recognitioncited by 1

50 most cited of 66 All 66 (JSON)