# Improving Domain-Specific ASR with LLM-Generated Contextual Descriptions

arXiv: https://arxiv.org/abs/2407.17874 · HTML: https://mlsanity.com/paper/2407.17874/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 18 references that are in the map.
- First posted on arXiv 25 Jul 2024.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion](https://mlsanity.com/paper/2104.02194/) (2021)
- [Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognition](https://mlsanity.com/paper/2306.16007/) (2023)
- [Contextualized End-to-End Speech Recognition with Contextual Phrase Prediction Network](https://mlsanity.com/paper/2305.12493/) (2023)
- [Can Contextual Biasing Remain Effective with Whisper and GPT-2?](https://mlsanity.com/paper/2306.01942/) (2023)
- [PromptASR for contextualized ASR with controllable style](https://mlsanity.com/paper/2309.07414/) (2023)
- [LyricWhiz: Robust Multilingual Zero-shot Lyrics Transcription by Whispering to ChatGPT](https://mlsanity.com/paper/2306.17103/) (2023)
- [CB-Conformer: Contextual biasing Conformer for biased word recognition](https://mlsanity.com/paper/2304.09607/) (2023)
- [Investigating Pre-trained Audio Encoders in the Low-Resource Condition](https://mlsanity.com/paper/2305.17733/) (2023)
- [Promptformer: Prompted Conformer Transducer for ASR](https://mlsanity.com/paper/2401.07360/) (2024)

## Built on by (most cited first, 6 of 6)

- [Retrieval-Augmented Speech Recognition Approach for Domain Challenges](https://mlsanity.com/paper/2502.15264/) (2025)
- [Zero-Shot Context-Aware ASR for Diverse Arabic Varieties](https://mlsanity.com/paper/2511.18774/) (2025)
- [Whisper: Courtside Edition Enhancing ASR Performance Through LLM-Driven Context Generation](https://mlsanity.com/paper/2602.18966/) (2026)
- [RECOVER: Robust Entity Correction via agentic Orchestration of hypothesis Variants for Evidence-based Recovery](https://mlsanity.com/paper/2603.16411/) (2026)
- [Elderly-Contextual Data Augmentation via Speech Synthesis for Elderly ASR](https://mlsanity.com/paper/2604.24770/) (2026)
- [agentic-ger: terminology recovery in long-form speech using global context](https://mlsanity.com/paper/2609.29428/) (2026)

## Abstract (arXiv)

End-to-end automatic speech recognition (E2E ASR) systems have significantly improved speech recognition through training on extensive datasets. Despite these advancements, they still struggle to accurately recognize domain specific words, such as proper nouns and technical terminologies. To address this problem, we propose a method to utilize the state-of-the-art Whisper without modifying its architecture, preserving its generalization performance while enabling it to leverage descriptions effectively. Moreover, we propose two additional training techniques to improve the domain specific ASR: decoder fine-tuning, and context perturbation. We also propose a method to use a Large Language Model (LLM) to generate descriptions with simple metadata, when descriptions are unavailable. Our experiments demonstrate that proposed methods notably enhance domain-specific ASR accuracy on real-life datasets, with LLM-generated descriptions outperforming human-crafted ones in effectiveness.