mlsanityData to 14 Oct 2026

Personalization for BERT-based Discriminative Speech Recognition Rescoring

arXiv 2307.06832 ↗First posted 13 Jul 2023eess.AS cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Recognition of personalized content remains a challenge in end-to-end speech recognition. We explore three novel approaches that use personalized content in a neural rescoring step to improve recognition: gazetteers, prompting, and a cross-attention based encoder-decoder model. We use internal de-identified en-US data from interactions with a virtual voice assistant supplemented with personalized named entities to compare these approaches. On a test set with personalized named entities, we show that each of these approaches improves word error rate by over 10%, against a neural rescoring baseline. We also show that on this test set, natural language prompts can improve word error rate by 7% without any training and with a marginal loss in generalization. Overall, gazetteers were found to perform the best with a 10% improvement in word error rate (WER), while also improving WER on a general test set by 1%.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 22 of its 33 references that are in the map. First posted on arXiv 13 Jul 2023.

Builds on

22 of 33 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  4. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 336
  5. 2022Training Language Models with Memory Augmentationcited by 113
  6. 2017Minimum Word Error Rate Training for Attention-based Sequence-to-Sequence Modelscited by 68
  7. 2020Deliberation Model Based Two-Pass End-to-End Speech Recognitioncited by 57
  8. 2022Contextual Adapters for Personalized Speech Recognition in Neural Transducerscited by 57
  9. 2021Context-Aware Transformer Transducer for Speech Recognitioncited by 56
  10. 2022RescoreBERT: Discriminative Speech Recognition Rescoring with BERTcited by 34
  11. 2021Transformer Based Deliberation for Two-Pass Speech Recognitioncited by 29
  12. 2021Personalization Strategies for End-to-End Speech Recognition Systemscited by 26
10 more
  1. 2021ASR Rescoring and Confidence Estimation with ELECTRAcited by 16
  2. 2019Audio-attention discriminative language model for ASR rescoringcited by 13
  3. 2021Domain-aware Neural Language Models for Speech Recognitioncited by 13
  4. 2019Scalable Multi Corpora Neural Language Models for ASRcited by 12
  5. 2021N-Best ASR Transformer: Enhancing SLU Performance using Multiple ASR Hypothesescited by 12
  6. 2022Improving Rare Word Recognition with LM-aware MWER Trainingcited by 12
  7. 2019An Empirical Study of Efficient ASR Rescoring with Transformerscited by 11
  8. 2021Lattention: Lattice-attention in ASR rescoringcited by 6
  9. 2022Joint Audio/Text Training for Transformer Rescorer of Streaming Speech Recognitioncited by 5
  10. 2020Neural Composition: Learning to Generate from Multiple Modelscited by 3

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2025Better Pseudo-labeling with Multi-ASR Fusion and Error Correction by SpeechLLMcited by 6
  2. 2024Speech Recognition Rescoring with Large Speech-Text Foundation Modelscited by 2