mlsanityData to 7 Oct 2026

Domain-aware Neural Language Models for Speech Recognition

arXiv 2101.03229 ↗First posted 5 Jan 2021cs.CL cs.SD eess.AScited by 13

Not on the map. Why?

Abstract (arXiv)

As voice assistants become more ubiquitous, they are increasingly expected to support and perform well on a wide variety of use-cases across different domains. We present a domain-aware rescoring framework suitable for achieving domain-adaptation during second-pass rescoring in production settings. In our framework, we fine-tune a domain-general neural language model on several domains, and use an LSTM-based domain classification model to select the appropriate domain-adapted model to use for second-pass rescoring. This domain-aware rescoring improves the word error rate by up to 2.4% and slot word error rate by up to 4.1% on three individual domains -- shopping, navigation, and music -- compared to domain general rescoring. These improvements are obtained while maintaining accuracy for the general use case.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 8 of its 26 references that are in the map. First posted on arXiv 5 Jan 2021.

Builds on

8 of 26 references.

References · 8

  1. 2016Exploring the Limits of Language Modelingcited by 172
  2. 2015End-to-End Attention-based Large Vocabulary Speech Recognitioncited by 148
  3. 2018Pseudo-Recursal: Solving the Catastrophic Forgetting Problem in Deep Neural Networkscited by 23
  4. 2019Audio-attention discriminative language model for ASR rescoringcited by 13
  5. 2019Scalable Multi Corpora Neural Language Models for ASRcited by 12
  6. 2018Contextual Language Model Adaptation for Conversational Agentscited by 10
  7. 2017Lattice Rescoring Strategies for Long Short Term Memory Language Models in Speech Recognitioncited by 9
  8. 2020Neural Composition: Learning to Generate from Multiple Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 1; 2023: 6; 2024: 1; 2025: 0; 2026: 1.

13 papers.

Sort:

By area:

  1. 2021Context-Aware Transformer Transducer for Speech Recognitioncited by 56
  2. 2023Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognitioncited by 56
  3. 2021Personalization Strategies for End-to-End Speech Recognition Systemscited by 26
  4. 2023Zero-shot Domain-sensitive Speech Recognition with Prompt-conditioning Fine-tuningcited by 13
  5. 2023Low-rank Adaptation of Large Language Model Rescoring for Parameter-Efficient Speech Recognitioncited by 13
  6. 2021Semantic-WER: A Unified Metric for the Evaluation of ASR Transcript for End Usabilitycited by 12
  7. 2021Cross-utterance Reranking Models with BERT and Graph Convolutional Networks for Conversational Speech Recognitioncited by 7
  8. 2022A Simple Baseline for Domain Adaptation in End to End ASR Systems Using Synthetic Datacited by 7
  9. 2023Dialog act guided contextual adapter for personalized speech recognitioncited by 4
  10. 2024Multi-Modal Retrieval For Large Language Model Based Speech Recognitioncited by 4
  11. 2023Personalization for BERT-based Discriminative Speech Recognition Rescoringcited by 2
  12. 2023Mask The Bias: Improving Domain-Adaptive Generalization of CTC-based ASR with Internal Language Model Estimationcited by 1
1 more
  1. 2026Listen to the Latents: Self-Correcting Speech Recognition in Large Audio Language Models Through Hidden-State Interactionscited by 0