mlsanityData to 14 Oct 2026

Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion

arXiv 2104.02194 ↗First posted 5 Apr 2021cs.CL cs.LG eess.AScited by 101

Not on the map. Why?

Abstract (arXiv)

How to leverage dynamic contextual information in end-to-end speech recognition has remained an active research area. Previous solutions to this problem were either designed for specialized use cases that did not generalize well to open-domain scenarios, did not scale to large biasing lists, or underperformed on rare long-tail words. We address these limitations by proposing a novel solution that combines shallow fusion, trie-based deep biasing, and neural network language model contextualization. These techniques result in significant 19.5% relative Word Error Rate improvement over existing contextual biasing approaches and 5.4%-9.3% improvement compared to a strong hybrid baseline on both open-domain and constrained contextualization tasks, where the targets consist of mostly rare long-tail words. Our final system remains lightweight and modular, allowing for quick modification without model re-training.

Key facts. Cited by 101 papers in the map (data through 14 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 5 Apr 2021.

Builds on

22 of 30 references.

References · 22

  1. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  4. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 639
  5. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 234
  6. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 150
  7. 2018Deep context: end-to-end contextual speech recognitioncited by 137
  8. 2020Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognitioncited by 113
  9. 2017An analysis of incorporating an external language model into a sequence-to-sequence modelcited by 111
  10. 2020Hybrid Autoregressive Transducer (hat)cited by 108
  11. 2017Cold Fusion: Training Seq2Seq Models Together with Language Modelscited by 88
  12. 2020Contextual RNN-T For Open Domain ASRcited by 85
10 more
  1. 2020Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognitioncited by 80
  2. 2020A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognitioncited by 73
  3. 2018A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognitioncited by 66
  4. 2020Deep Shallow Fusion for RNN-T Personalizationcited by 64
  5. 2020Alignment Restricted Streaming Recurrent Neural Network Transducercited by 42
  6. 2020Improving RNN Transducer Based ASR with Auxiliary Taskscited by 27
  7. 2020Improved Neural Language Model Fusion for Streaming Recurrent Neural Network Transducercited by 22
  8. 2019From Senones to Chenones: Tied Context-Dependent Graphemes for Hybrid Speech Recognitioncited by 20
  9. 2019G2G: TTS-Driven Pronunciation Learning for Graphemic Hybrid ASRcited by 11
  10. 2020Benchmarking LF-MMI, CTC and RNN-T Criteria for Streaming ASRcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 6; 2022: 14; 2023: 22; 2024: 21; 2025: 20; 2026: 18.

101 papers.

Sort:

By area:

  1. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 140
  2. 2024OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformercited by 74
  3. 2022Contextual Adapters for Personalized Speech Recognition in Neural Transducerscited by 57
  4. 2022Improving End-to-End Contextual Speech Recognition with Fine-Grained Contextual Knowledge Selectioncited by 39
  5. 2021Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognitioncited by 32
  6. 2023Contextualized End-to-End Speech Recognition with Contextual Phrase Prediction Networkcited by 31
  7. 2023COSMIC: Data Efficient Instruction-tuning For Speech In-Context Learningcited by 31
  8. 2023End-to-End Speech Recognition Contextualization with Large Language Modelscited by 22
  9. 2021Evaluating User Perception of Speech Recognition System Quality with Semantic Distance Metriccited by 21
  10. 2023Can Contextual Biasing Remain Effective with Whisper and GPT-2?cited by 21
  11. 2023PromptASR for contextualized ASR with controllable stylecited by 19
  12. 2024Contextualized Automatic Speech Recognition with Attention-Based Bias Phrase Boosted Beam Searchcited by 19
38 more
  1. 2024CTC-Assisted LLM-Based Contextual ASRcited by 19
  2. 2023Robust Acoustic and Semantic Contextual Biasing in Neural Transducers for Speech Recognitioncited by 18
  3. 2022Towards Contextual Spelling Correction for Customization of End-to-end Speech Recognition Systemscited by 17
  4. 2022Tree-constrained Pointer Generator with Graph Neural Network Encodings for Contextual Speech Recognitioncited by 17
  5. 2022Improving Contextual Recognition of Rare Words with an Alternate Spelling Prediction Modelcited by 17
  6. 2024Improving ASR Contextual Biasing with Guided Attentioncited by 17
  7. 2025ContextASR-Bench: A Massive Contextual Speech Recognition Benchmarkcited by 15
  8. 2023Efficient Domain Adaptation for Speech Foundation Modelscited by 13
  9. 2023Phoneme-aware Encoding for Prefix-tree-based Contextual ASRcited by 13
  10. 2023Adaptive Contextual Biasing for Transducer Based Streaming Speech Recognitioncited by 12
  11. 2022Improving the fusion of acoustic and text representations in RNN-Tcited by 11
  12. 2022Minimising Biasing Word Errors for Contextual ASR with the Tree-Constrained Pointer Generatorcited by 11
  13. 2022Fast and accurate factorized neural transducer for text adaption of end-to-end speech recognition modelscited by 11
  14. 2024Improving Neural Biasing for Contextual Speech Recognition by Early Context Injection and Text Perturbationcited by 11
  15. 2021Spell my name: keyword boosted speech recognitioncited by 10
  16. 2024Contextualized End-to-end Automatic Speech Recognition with Intermediate Biasing Losscited by 10
  17. 2022A Language Agnostic Multilingual Streaming On-Device ASR Systemcited by 9
  18. 2023Improving Contextual Spelling Correction by External Acoustics Attention and Semantic Aware Data Augmentationcited by 9
  19. 2023Contextual Biasing with the Knuth-Morris-Pratt Matching Algorithmcited by 9
  20. 2022Deliberation Model for On-Device Spoken Language Understandingcited by 8
  21. 2024Contextualized Automatic Speech Recognition with Dynamic Vocabularycited by 8
  22. 2026SLAM-LLM: A Modular, Open-Source Multimodal Large Language Model Framework and Best Practice for Speech, Language, Audio and Music Processingcited by 8
  23. 2021On lattice-free boosted MMI training of HMM and CTC-based full-context ASR modelscited by 7
  24. 2023Contextual Biasing of Named-Entities with Large Language Modelscited by 7
  25. 2023ed-cec: improving rare word recognition using asr postprocessing based on error detection and context-aware error correctioncited by 7
  26. 2024Fast Context-Biasing for CTC and Transducer ASR models with CTC-based Word Spottercited by 7
  27. 2022Neural-FST Class Language Model for End-to-End Speech Recognitioncited by 6
  28. 2022UserLibri: A Dataset for ASR Personalization Using Only Textcited by 6
  29. 2022End-to-end Spoken Language Understanding with Tree-constrained Pointer Generatorcited by 6
  30. 2023Graph Neural Networks for Contextual ASR with the Tree-Constrained Pointer Generatorcited by 6
  31. 2023Improving Large-scale Deep Biasing with Phoneme Features and Text-only Data in Streaming Transducercited by 6
  32. 2024Improving Domain-Specific ASR with LLM-Generated Contextual Descriptionscited by 6
  33. 2023Text Generation with Speech Synthesis for ASR Data Augmentationcited by 5
  34. 2024Keyword-Guided Adaptation of Automatic Speech Recognitioncited by 5
  35. 2024InterBiasing: Boost Unseen Word Recognition through Biasing Intermediate Predictionscited by 5
  36. 2024Contextual Biasing to Improve Domain-specific Custom Vocabulary Audio Transcription without Explicit Fine-Tuning of Whisper Modelcited by 5
  37. 2024M3AV: A Multimodal, Multigenre, and Multipurpose Audio-Visual Academic Lecture Datasetcited by 4
  38. 2024Let's Fuse Step by Step: A Generative Fusion Decoding Algorithm with LLMs for Robust and Instruction-Aware ASR and OCRcited by 4

50 most cited of 101 All 101 (JSON)