# An Effective Context-Balanced Adaptation Approach for Long-Tailed Speech Recognition

arXiv: https://arxiv.org/abs/2409.06468 · HTML: https://mlsanity.com/paper/2409.06468/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 24 references that are in the map.
- First posted on arXiv 10 Sep 2024.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Deep context: end-to-end contextual speech recognition](https://mlsanity.com/paper/1808.02480/) (2018)
- [On Using Very Large Target Vocabulary for Neural Machine Translation](https://mlsanity.com/paper/1412.2007/) (2014)
- [Contextual Adapters for Personalized Speech Recognition in Neural Transducers](https://mlsanity.com/paper/2205.13660/) (2022)
- [Adapt-and-Adjust: Overcoming the Long-Tail Problem of Multilingual Speech Recognition](https://mlsanity.com/paper/2012.01687/) (2020)
- [Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognition](https://mlsanity.com/paper/2109.00627/) (2021)
- [Personalization Strategies for End-to-End Speech Recognition Systems](https://mlsanity.com/paper/2102.07739/) (2021)
- [Attention based on-device streaming speech recognition with large speech corpus](https://mlsanity.com/paper/2001.00577/) (2020)
- [Improving ASR Contextual Biasing with Guided Attention](https://mlsanity.com/paper/2401.08835/) (2024)
- [No Need for a Lexicon? Evaluating the Value of the Pronunciation Lexica in End-to-End Models](https://mlsanity.com/paper/1712.01864/) (2017)
- [Phoneme-aware Encoding for Prefix-tree-based Contextual ASR](https://mlsanity.com/paper/2312.09582/) (2023)
- [CB-Conformer: Contextual biasing Conformer for biased word recognition](https://mlsanity.com/paper/2304.09607/) (2023)
- [PROCTER: PROnunciation-aware ConTextual adaptER for personalized speech recognition in neural transducers](https://mlsanity.com/paper/2303.17131/) (2023)
- [DANCER: Entity Description Augmented Named Entity Corrector for Automatic Speech Recognition](https://mlsanity.com/paper/2403.17645/) (2024)

## Built on by (most cited first, 1 of 1)

- [A Neural Model for Contextual Biasing Score Learning and Filtering](https://mlsanity.com/paper/2510.23849/) (2025)

## Abstract (arXiv)

End-to-end (E2E) automatic speech recognition (ASR) models have become standard practice for various commercial applications. However, in real-world scenarios, the long-tailed nature of word distribution often leads E2E ASR models to perform well on common words but fall short in recognizing uncommon ones. Recently, the notion of a contextual adapter (CA) was proposed to infuse external knowledge represented by a context word list into E2E ASR models. Although CA can improve recognition performance on rare words, two crucial data imbalance problems remain. First, when using low-frequency words as context words during training, since these words rarely occur in the utterance, CA becomes prone to overfit on attending to the <no-context> token due to higher-frequency words not being present in the context list. Second, the long-tailed distribution within the context list itself still causes the model to perform poorly on low-frequency context words. In light of this, we explore in-depth the impact of altering the context list to have words with different frequency distributions on model performance, and meanwhile extend CA with a simple yet effective context-balanced learning objective. A series of experiments conducted on the AISHELL-1 benchmark dataset suggests that using all vocabulary words from the training corpus as the context list and pairing them with our balanced objective yields the best performance, demonstrating a significant reduction in character error rate (CER) by up to 1.21% and a more pronounced 9.44% reduction in the error rate of zero-shot words.