mlsanityData to 14 Oct 2026

NGPU-LM: GPU-Accelerated N-Gram Language Model for Context-Biasing in Greedy ASR Decoding

arXiv 2505.22857 ↗First posted 28 May 2025eess.AS cs.AI cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Statistical n-gram language models are widely used for context-biasing tasks in Automatic Speech Recognition (ASR). However, existing implementations lack computational efficiency due to poor parallelization, making context-biasing less appealing for industrial use. This work rethinks data structures for statistical n-gram language models to enable fast and parallel operations for GPU-optimized inference. Our approach, named NGPU-LM, introduces customizable greedy decoding for all major ASR model types - including transducers, attention encoder-decoder models, and CTC - with less than 7% computational overhead. The proposed approach can eliminate more than 50% of the accuracy gap between greedy and beam search for out-of-domain scenarios while avoiding significant slowdown caused by beam search. The implementation of the proposed NGPU-LM is open-sourced.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 26 of its 38 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

26 of 38 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  5. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  6. 2015Listen, Attend and Spellcited by 543
  7. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  8. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  9. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 235
  10. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 198
  11. 2018Deep context: end-to-end contextual speech recognitioncited by 137
  12. 2023Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognitioncited by 124
14 more
  1. 2020Hybrid Autoregressive Transducer (hat)cited by 108
  2. 2020Contextual RNN-T For Open Domain ASRcited by 85
  3. 2020Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognitioncited by 80
  4. 2021SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognitioncited by 78
  5. 2023SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translationcited by 76
  6. 2020A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognitioncited by 73
  7. 2023Efficient Sequence Transduction by Jointly Predicting Tokens and Durationscited by 58
  8. 2022Flashlight: Enabling Innovation in Tools for Machine Learningcited by 25
  9. 2022Modular Hybrid Autoregressive Transducercited by 17
  10. 2020Accelerating RNN Transducer Inference via One-Step Constrained Beam Searchcited by 13
  11. 2024Speed of Light Exact Greedy Decoding for RNN-T Speech Recognition Models on GPUcited by 12
  12. 2021Spell my name: keyword boosted speech recognitioncited by 10
  13. 2024Label-Looping: Highly Efficient Decoding for Transducerscited by 9
  14. 2024Fast Context-Biasing for CTC and Transducer ASR models with CTC-based Word Spottercited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025ContextASR-Bench: A Massive Contextual Speech Recognition Benchmarkcited by 15
  2. 2025Pushing the Limits of Beam Search Decoding for Transducer-based ASR modelscited by 4
  3. 2025TurboBias: Universal ASR Context-Biasing powered by GPU-accelerated Phrase-Boosting Treecited by 4
  4. 2025Supplementary Resources and Analysis for Automatic Speech Recognition Systems Trained on the Loquacious Datasetcited by 2
  5. 2025FlexCTC: GPU-powered CTC Beam Decoding With Advanced Contextual Abilitiescited by 1
  6. 2026TurboBias 2.0: Streaming Context-Biasing for Production-Efficient ASR Systemscited by 0
  7. 2026Typhoon ASR Streaming: Steerable Low-Latency Thai Speech Recognition with Real-Time Shallow Fusioncited by 0