mlsanityData to 14 Oct 2026

Contextual Adapters for Personalized Speech Recognition in Neural Transducers

arXiv 2205.13660 ↗First posted 26 May 2022cs.CL cs.LGcited by 57

Not on the map. Why?

Abstract (arXiv)

Personal rare word recognition in end-to-end Automatic Speech Recognition (E2E ASR) models is a challenge due to the lack of training data. A standard way to address this issue is with shallow fusion methods at inference time. However, due to their dependence on external language models and the deterministic approach to weight boosting, their performance is limited. In this paper, we propose training neural contextual adapters for personalization in neural transducer based ASR models. Our approach can not only bias towards user-defined words, but also has the flexibility to work with pretrained ASR models. Using an in-house dataset, we demonstrate that contextual adapters can be applied to any general purpose pretrained ASR model to improve personalization. Our method outperforms shallow fusion, while retaining functionality of the pretrained models by not altering any of the model weights. We further show that the adapter style training is superior to full-fine-tuning of the ASR models on datasets with user-defined content.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). Builds on 19 of its 26 references that are in the map. First posted on arXiv 26 May 2022.

Builds on

19 of 26 references.

References · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  5. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  6. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 639
  7. 2017Learning multiple visual domains with residual adapterscited by 557
  8. 2015Listen, Attend and Spellcited by 543
  9. 2015Attention-Based Models for Speech Recognitioncited by 401
  10. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 234
  11. 2018Efficient parametrization of multi-domain deep neural networkscited by 168
  12. 2018Deep context: end-to-end contextual speech recognitioncited by 137
7 more
  1. 2021Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusioncited by 101
  2. 2020Contextual RNN-T For Open Domain ASRcited by 85
  3. 2020A Better and Faster End-to-End Model for Streaming ASRcited by 69
  4. 2020Deep Shallow Fusion for RNN-T Personalizationcited by 64
  5. 2021Context-Aware Transformer Transducer for Speech Recognitioncited by 56
  6. 2021Personalization Strategies for End-to-End Speech Recognition Systemscited by 26
  7. 2017No Need for a Lexicon? Evaluating the Value of the Pronunciation Lexica in End-to-End Modelscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 26; 2024: 16; 2025: 8; 2026: 5.

57 papers.

Sort:

By area:

  1. 2023Modular Deep Learningcited by 83
  2. 2023End-to-End Speech Recognition Contextualization with Large Language Modelscited by 22
  3. 2023Robust Acoustic and Semantic Contextual Biasing in Neural Transducers for Speech Recognitioncited by 18
  4. 2023SeACo-Paraformer: A Non-Autoregressive ASR System with Flexible and Effective Hotword Customization Abilitycited by 18
  5. 2024Improving ASR Contextual Biasing with Guided Attentioncited by 17
  6. 2023CopyNE: Better Contextual ASR by Copying Named Entitiescited by 14
  7. 2023Zero-shot Domain-sensitive Speech Recognition with Prompt-conditioning Fine-tuningcited by 13
  8. 2023Phoneme-aware Encoding for Prefix-tree-based Contextual ASRcited by 13
  9. 2023CB-Conformer: Contextual biasing Conformer for biased word recognitioncited by 11
  10. 2024Improving Neural Biasing for Contextual Speech Recognition by Early Context Injection and Text Perturbationcited by 11
  11. 2024Contextualized End-to-end Automatic Speech Recognition with Intermediate Biasing Losscited by 10
  12. 2023Approximate Nearest Neighbour Phrase Mining for Contextual Speech Recognitioncited by 8
38 more
  1. 2023Retrieve and Copy: Scaling ASR Personalization to Large Catalogscited by 7
  2. 2023Improving Large-scale Deep Biasing with Phoneme Features and Text-only Data in Streaming Transducercited by 6
  3. 2023Using External Off-Policy Speech-To-Text Mappings in Contextual End-To-End Automated Speech Recognitioncited by 5
  4. 2023Retraining-free Customized ASR for Enharmonic Words Based on a Named-Entity-Aware Model and Phoneme Similarity Estimationcited by 5
  5. 2023A Multitask Training Approach to Enhance Whisper with Contextual Biasing and Open-Vocabulary Keyword Spottingcited by 5
  6. 2024Deferred NAM: Low-latency Top-K Context Injection via Deferred Context Encoding for Non-Streaming ASRcited by 5
  7. 2024Keyword-Guided Adaptation of Automatic Speech Recognitioncited by 5
  8. 2023On-the-fly Text Retrieval for End-to-End ASR Adaptationcited by 4
  9. 2023Dialog act guided contextual adapter for personalized speech recognitioncited by 4
  10. 2023Multilingual Contextual Adapters To Improve Custom Word Recognition In Low-resource Languagescited by 4
  11. 2024Exploration of Adapter for Noise Robust Automatic Speech Recognitioncited by 4
  12. 2024DANCER: Entity Description Augmented Named Entity Corrector for Automatic Speech Recognitioncited by 4
  13. 2024Multi-Modal Retrieval For Large Language Model Based Speech Recognitioncited by 4
  14. 2023Leveraging Pretrained ASR Encoders for Effective and Efficient End-to-End Speech Intent Classification and Slot Fillingcited by 3
  15. 2024Promptformer: Prompted Conformer Transducer for ASRcited by 3
  16. 2024Locality enhanced dynamic biasing and sampling strategies for contextual ASRcited by 3
  17. 2024Optimizing Contextual Speech Recognition Using Vector Quantization for Efficient Retrievalcited by 3
  18. 2025WCTC-Biasing: Retraining-free Contextual Biasing ASR with Wildcard CTC-based Keyword Spotting and Inter-layer Biasingcited by 3
  19. 2023Dual-Attention Neural Transducers for Efficient Wake Word Spotting in Speech Recognitioncited by 2
  20. 2023Improving Code-Switching and Named Entity Recognition in ASR with Speech Editing based Data Augmentationcited by 2
  21. 2023Personalization for BERT-based Discriminative Speech Recognition Rescoringcited by 2
  22. 2022Damage Control During Domain Adaptation for Transducer Based Automatic Speech Recognitioncited by 1
  23. 2023Adaptive Endpointing with Deep Contextual Multi-armed Banditscited by 1
  24. 2024Task Oriented Dialogue as a Catalyst for Self-Supervised Automatic Speech Recognitioncited by 1
  25. 2024Self-consistent context aware conformer transducer for speech recognitioncited by 1
  26. 2024Leveraging Parameter-Efficient Transfer Learning for Multi-Lingual Text-to-Speech Adaptationcited by 1
  27. 2024An Effective Context-Balanced Adaptation Approach for Long-Tailed Speech Recognitioncited by 1
  28. 2025Lightweight Prompt Biasing for Contextualized End-to-End ASR Systemscited by 1
  29. 2025PMF-CEC: Phoneme-augmented Multimodal Fusion for Context-aware ASR Error Correction with Error-specific Selective Decodingcited by 1
  30. 2025Improving Synthetic Data Training for Contextual Biasing Models with a Keyword-Aware Cost Functioncited by 1
  31. 2025Retrieval Augmented Generation based context discovery for ASRcited by 1
  32. 2025A Neural Model for Contextual Biasing Score Learning and Filteringcited by 1
  33. 2022Towards Personalization of CTC Speech Recognition Models with Contextual Adapters and Adaptive Boostingcited by 0
  34. 2023Streaming Speech-to-Confusion Network Speech Recognitioncited by 0
  35. 2023Incorporating Class-based Language Model for Named Entity Recognition in Factorized Neural Transducercited by 0
  36. 2023Wiki-En-ASR-Adapt: Large-scale synthetic dataset for English ASR Customizationcited by 0
  37. 2023Cascaded Multi-task Adaptive Learning Based on Neural Architecture Searchcited by 0
  38. 2024An Efficient Self-Learning Framework For Interactive Spoken Dialog Systemscited by 0

50 most cited of 57 All 57 (JSON)