# Personalization for BERT-based Discriminative Speech Recognition Rescoring

arXiv: https://arxiv.org/abs/2307.06832 · HTML: https://mlsanity.com/paper/2307.06832/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 13 Jul 2023.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery](https://mlsanity.com/paper/2302.03668/) (2023)
- [Training Language Models with Memory Augmentation](https://mlsanity.com/paper/2205.12674/) (2022)
- [Minimum Word Error Rate Training for Attention-based Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.01818/) (2017)
- [Deliberation Model Based Two-Pass End-to-End Speech Recognition](https://mlsanity.com/paper/2003.07962/) (2020)
- [Contextual Adapters for Personalized Speech Recognition in Neural Transducers](https://mlsanity.com/paper/2205.13660/) (2022)
- [Context-Aware Transformer Transducer for Speech Recognition](https://mlsanity.com/paper/2111.03250/) (2021)
- [RescoreBERT: Discriminative Speech Recognition Rescoring with BERT](https://mlsanity.com/paper/2202.01094/) (2022)
- [Transformer Based Deliberation for Two-Pass Speech Recognition](https://mlsanity.com/paper/2101.11577/) (2021)
- [Personalization Strategies for End-to-End Speech Recognition Systems](https://mlsanity.com/paper/2102.07739/) (2021)
- [ASR Rescoring and Confidence Estimation with ELECTRA](https://mlsanity.com/paper/2110.01857/) (2021)
- [Audio-attention discriminative language model for ASR rescoring](https://mlsanity.com/paper/1912.03363/) (2019)
- [Domain-aware Neural Language Models for Speech Recognition](https://mlsanity.com/paper/2101.03229/) (2021)
- [Scalable Multi Corpora Neural Language Models for ASR](https://mlsanity.com/paper/1907.01677/) (2019)
- [N-Best ASR Transformer: Enhancing SLU Performance using Multiple ASR Hypotheses](https://mlsanity.com/paper/2106.06519/) (2021)
- [Improving Rare Word Recognition with LM-aware MWER Training](https://mlsanity.com/paper/2204.07553/) (2022)
- [An Empirical Study of Efficient ASR Rescoring with Transformers](https://mlsanity.com/paper/1910.11450/) (2019)
- [Lattention: Lattice-attention in ASR rescoring](https://mlsanity.com/paper/2111.10157/) (2021)
- [Joint Audio/Text Training for Transformer Rescorer of Streaming Speech Recognition](https://mlsanity.com/paper/2211.00174/) (2022)
- [Neural Composition: Learning to Generate from Multiple Models](https://mlsanity.com/paper/2007.16013/) (2020)

## Built on by (most cited first, 2 of 2)

- [Better Pseudo-labeling with Multi-ASR Fusion and Error Correction by SpeechLLM](https://mlsanity.com/paper/2506.11089/) (2025)
- [Speech Recognition Rescoring with Large Speech-Text Foundation Models](https://mlsanity.com/paper/2409.16654/) (2024)

## Abstract (arXiv)

Recognition of personalized content remains a challenge in end-to-end speech recognition. We explore three novel approaches that use personalized content in a neural rescoring step to improve recognition: gazetteers, prompting, and a cross-attention based encoder-decoder model. We use internal de-identified en-US data from interactions with a virtual voice assistant supplemented with personalized named entities to compare these approaches. On a test set with personalized named entities, we show that each of these approaches improves word error rate by over 10%, against a neural rescoring baseline. We also show that on this test set, natural language prompts can improve word error rate by 7% without any training and with a marginal loss in generalization. Overall, gazetteers were found to perform the best with a 10% improvement in word error rate (WER), while also improving WER on a general test set by 1%.