mlsanityData to 7 Oct 2026

An analysis of incorporating an external language model into a sequence-to-sequence model

arXiv 1712.01996 ↗First posted 6 Dec 2017eess.AS cs.AI cs.CLcited by 109

Not on the map. Why?

Abstract (arXiv)

Attention-based sequence-to-sequence models for automatic speech recognition jointly train an acoustic model, language model, and alignment mechanism. Thus, the language model component is only trained on transcribed audio-text pairs. This leads to the use of shallow fusion with an external language model at inference time. Shallow fusion refers to log-linear interpolation with a separately trained language model at each step of the beam search. In this work, we investigate the behavior of shallow fusion across a range of conditions: different types of language models, different decoding units, and different tasks. On Google Voice Search, we demonstrate that the use of shallow fusion with a neural LM with wordpieces yields a 9.1% relative word error rate reduction (WERR) over our competitive attention-based sequence-to-sequence model, obviating the need for second-pass rescoring.

Key facts. Cited by 109 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 33; 2023: 29; 2024: 17; 2025: 9; 2026: 6.

109 papers.

Sort:

By area:

  1. 2023End-to-End Speech Recognition: A Surveycited by 127
  2. 2021Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusioncited by 100
  3. 2021Sub-word Level Lip Reading With Visual Attentioncited by 60
  4. 2021Context-Aware Transformer Transducer for Speech Recognitioncited by 56
  5. 2023Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognitioncited by 48
  6. 2023HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Modelscited by 41
  7. 2021Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognitioncited by 32
  8. 2021Factorized Neural Transducer for Efficient Language Model Adaptationcited by 31
  9. 2021Personalization Strategies for End-to-End Speech Recognition Systemscited by 26
  10. 2022Scaling Language Model Size in Cross-Device Federated Learningcited by 25
  11. 2022JOIST: A Joint Speech and Text Streaming Model For ASRcited by 24
  12. 2023An Overview on Language Models: Recent Developments and Outlookcited by 22
38 more
  1. 2024It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognitioncited by 22
  2. 2022Deep Learning for Visual Speech Analysis: A Surveycited by 21
  3. 2021Fast Text-Only Domain Adaptation of RNN-Transducer Prediction Networkcited by 19
  4. 2022AISHELL-NER: Named Entity Recognition from Chinese Speechcited by 19
  5. 2024Contextualized Automatic Speech Recognition with Attention-Based Bias Phrase Boosted Beam Searchcited by 19
  6. 2023PromptASR for contextualized ASR with controllable stylecited by 18
  7. 2021ASR Rescoring and Confidence Estimation with ELECTRAcited by 16
  8. 2022Effectiveness of Mining Audio and Text Pairs from Public Data for Improving ASR Systems for Low-Resource Languagescited by 16
  9. 2021Improving Hybrid CTC/Attention End-to-end Speech Recognition with Pretrained Acoustic and Language Modelcited by 14
  10. 2023Large-scale Language Model Rescoring on Long-form Datacited by 14
  11. 2022Maestro-U: Leveraging joint speech-text representation learning for zero supervised speech ASRcited by 13
  12. 2023Multi-stage Large Language Model Correction for Speech Recognitioncited by 13
  13. 2022Improving Rare Word Recognition with LM-aware MWER Trainingcited by 12
  14. 2024Multilingual and Fully Non-Autoregressive ASR with Large Language Model Fusion: A Comprehensive Studycited by 12
  15. 2021Timers and Such: A Practical Benchmark for Spoken Language Understanding with Numberscited by 11
  16. 2022Minimising Biasing Word Errors for Contextual ASR with the Tree-Constrained Pointer Generatorcited by 11
  17. 2022BERT Meets CTC: New Formulation of End-to-End Speech Recognition with Pre-trained Masked Language Modelcited by 11
  18. 2022Fast and accurate factorized neural transducer for text adaption of end-to-end speech recognition modelscited by 11
  19. 2023CB-Conformer: Contextual biasing Conformer for biased word recognitioncited by 11
  20. 2023Massively Multilingual Shallow Fusion with Large Language Modelscited by 10
  21. 2021Lookup-Table Recurrent Language Models for Long Tail Speech Recognitioncited by 9
  22. 2022Internal Language Model Estimation Through Explicit Context Vector Learning for Attention-based Encoder-decoder ASRcited by 9
  23. 2022Detecting Unintended Memorization in Language-Model-Fused ASRcited by 9
  24. 2023Recovering from Privacy-Preserving Masking with Large Language Modelscited by 9
  25. 2024Failing Forward: Improving Generative Error Correction for ASR with Synthetic Data and Retrieval Augmentationcited by 9
  26. 2023Blank-regularized CTC for Frame Skipping in Neural Transducercited by 8
  27. 2023Improving Seq2Seq Grammatical Error Correction via Decoding Interventionscited by 8
  28. 2024Contextualized Automatic Speech Recognition with Dynamic Vocabularycited by 8
  29. 2024SALSA: Speedy ASR-LLM Synchronous Aggregationcited by 8
  30. 2021Language model fusion for streaming end to end speech recognitioncited by 7
  31. 2022End-to-end contextual asr based on posterior distribution adaptation for hybrid ctc/attention systemcited by 7
  32. 2022A Simple Baseline for Domain Adaptation in End to End ASR Systems Using Synthetic Datacited by 7
  33. 2023Exploring the Integration of Large Language Models into Automatic Speech Recognition Systems: An Empirical Studycited by 7
  34. 2023A Novel Self-training Approach for Low-resource Speech Recognitioncited by 7
  35. 2023Retrieve and Copy: Scaling ASR Personalization to Large Catalogscited by 7
  36. 2022Neural-FST Class Language Model for End-to-End Speech Recognitioncited by 6
  37. 2022UserLibri: A Dataset for ASR Personalization Using Only Textcited by 6
  38. 2022Non-autoregressive Error Correction for CTC-based ASR with Phone-conditioned Masked LMcited by 6

50 most cited of 109 All 109 (JSON)