mlsanityData to 7 Oct 2026

Improving Rare Word Recognition with LM-aware MWER Training

arXiv 2204.07553 ↗First posted 15 Apr 2022cs.CL cs.SD eess.AScited by 12

Not on the map. Why?

Abstract (arXiv)

Language models (LMs) significantly improve the recognition accuracy of end-to-end (E2E) models on words rarely seen during training, when used in either the shallow fusion or the rescoring setups. In this work, we introduce LMs in the learning of hybrid autoregressive transducer (HAT) models in the discriminative training framework, to mitigate the training versus inference gap regarding the use of LMs. For the shallow fusion setup, we use LMs during both hypotheses generation and loss computation, and the LM-aware MWER-trained model achieves 10% relative improvement over the model trained with standard MWER on voice search test sets containing rare words. For the rescoring setup, we learn a small neural module to generate per-token fusion weights in a data-dependent manner. This model achieves the same rescoring WER as regular MWER-trained model, but without the need for sweeping fusion weights.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 23 of its 35 references that are in the map. First posted on arXiv 15 Apr 2022.

Builds on

23 of 35 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  4. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  5. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 149
  6. 2017An analysis of incorporating an external language model into a sequence-to-sequence modelcited by 109
  7. 2020Hybrid Autoregressive Transducer (hat)cited by 106
  8. 2016Towards better decoding and language model integration in sequence to sequence modelscited by 100
  9. 2017Cold Fusion: Training Seq2Seq Models Together with Language Modelscited by 87
  10. 2020Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognitioncited by 80
  11. 2019Recognizing long-form speech using streaming end-to-end modelscited by 77
  12. 2020A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognitioncited by 73
11 more
  1. 2017Minimum Word Error Rate Training for Attention-based Sequence-to-Sequence Modelscited by 68
  2. 2019SpecAugment on Large Scale Datasetscited by 60
  3. 2021Tied & Reduced RNN-T Decodercited by 35
  4. 2020Efficient minimum word error rate training of RNN-Transducer for end-to-end speech recognitioncited by 34
  5. 2020Less Is More: Improved RNN-T Decoding Using Limited Label Context and Path Mergingcited by 26
  6. 2020Improving Tail Performance of a Deliberation E2E ASR Model Using a Large Text Corpuscited by 24
  7. 2019Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognitioncited by 23
  8. 2017Optimizing expected word error rate via sampling for speech recognitioncited by 21
  9. 2021Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognitioncited by 19
  10. 2020On Minimum Word Error Rate Training of the Hybrid Autoregressive Transducercited by 11
  11. 2022Sentence-Select: Large-Scale Language Model Data Selection for Rare-Word Speech Recognitioncited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 7; 2024: 3; 2025: 0; 2026: 1.

12 papers.

  1. 2023Exploring the Integration of Large Language Models into Automatic Speech Recognition Systems: An Empirical Studycited by 7
  2. 2023Improving Large-scale Deep Biasing with Phoneme Features and Text-only Data in Streaming Transducercited by 6
  3. 2024Effective internal language model training and fusion for factorized transducer modelcited by 3
  4. 2024Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approachcited by 3
  5. 2023Personalization for BERT-based Discriminative Speech Recognition Rescoringcited by 2
  6. 2023Correction Focused Language Model Training for Speech Recognitioncited by 2
  7. 2023Server-side Rescoring of Spoken Entity-centric Knowledge Queries for Virtual Assistantscited by 2
  8. 2022SAN: a robust end-to-end ASR model architecturecited by 1
  9. 2023Improving Rare Words Recognition through Homophone Extension and Unified Writing for Low-resource Cantonese Speech Recognitioncited by 1
  10. 2024Spelling Correction through Rewriting of Non-Autoregressive ASR Latticescited by 1
  11. 2023O-1: Self-training with Oracle and 1-best Hypothesiscited by 0
  12. 2026Transcribe, Translate, and Optimize: Joint Reward Learning for Speech Translationcited by 0