# PromptASR for contextualized ASR with controllable style

arXiv: https://arxiv.org/abs/2309.07414 · HTML: https://mlsanity.com/paper/2309.07414/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 14 Sep 2023.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [An analysis of incorporating an external language model into a sequence-to-sequence model](https://mlsanity.com/paper/1712.01996/) (2017)
- [Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion](https://mlsanity.com/paper/2104.02194/) (2021)
- [Recognizing long-form speech using streaming end-to-end models](https://mlsanity.com/paper/1910.11455/) (2019)
- [Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognition](https://mlsanity.com/paper/2306.16007/) (2023)
- [Pruned RNN-T for fast, memory-efficient ASR training](https://mlsanity.com/paper/2206.13236/) (2022)
- [Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognition](https://mlsanity.com/paper/2109.00627/) (2021)
- [Contextualized End-to-End Speech Recognition with Contextual Phrase Prediction Network](https://mlsanity.com/paper/2305.12493/) (2023)
- [Adapting GPT, GPT-2 and BERT Language Models for Speech Recognition](https://mlsanity.com/paper/2108.07789/) (2021)
- [Attentive Contextual Carryover for Multi-Turn End-to-End Spoken Language Understanding](https://mlsanity.com/paper/2112.06743/) (2021)

## Built on by (most cited first, 19 of 19)

- [Chain-of-Thought Prompting for Speech Translation](https://mlsanity.com/paper/2409.11538/) (2024)
- [Improving Neural Biasing for Contextual Speech Recognition by Early Context Injection and Text Perturbation](https://mlsanity.com/paper/2407.10303/) (2024)
- [Fast Context-Biasing for CTC and Transducer ASR models with CTC-based Word Spotter](https://mlsanity.com/paper/2406.07096/) (2024)
- [Improving Domain-Specific ASR with LLM-Generated Contextual Descriptions](https://mlsanity.com/paper/2407.17874/) (2024)
- [OWSM-Biasing: Contextualizing Open Whisper-Style Speech Models for Automatic Speech Recognition with Dynamic Vocabulary](https://mlsanity.com/paper/2506.09448/) (2025)
- [TurboBias: Universal ASR Context-Biasing powered by GPU-accelerated Phrase-Boosting Tree](https://mlsanity.com/paper/2508.07014/) (2025)
- [Optimizing Contextual Speech Recognition Using Vector Quantization for Efficient Retrieval](https://mlsanity.com/paper/2411.00664/) (2024)
- [PROFASR-BENCH: A Benchmark for Context-Conditioned ASR in High-Stakes Professional Speech](https://mlsanity.com/paper/2512.23686/) (2025)
- [Lightweight Prompt Biasing for Contextualized End-to-End ASR Systems](https://mlsanity.com/paper/2506.06252/) (2025)
- [Enhancing the Robustness of Contextual ASR to Varying Biasing Information Volumes Through Purified Semantic Correlation Joint Modeling](https://mlsanity.com/paper/2509.05908/) (2025)
- [How to Recognize New Words: A Comparison Between Context Biasing Methods and Speech LLMs](https://mlsanity.com/paper/2608.05759/) (2026)
- [Bi-directional Context-Enhanced Speech Large Language Models for Multilingual Conversational ASR](https://mlsanity.com/paper/2506.13396/) (2025)
- [Context Biasing for Pronunciation-Orthography Mismatch in Automatic Speech Recognition](https://mlsanity.com/paper/2506.18703/) (2025)
- [Attention2Probability: Attention-Driven Terminology Probability Estimation for Robust Speech-to-Text System](https://mlsanity.com/paper/2508.18701/) (2025)
- [TokenVerse++: Towards Flexible Multitask Learning with Dynamic Task Activation](https://mlsanity.com/paper/2508.19856/) (2025)
- [Enhancing Fully Formatted End-to-End Speech Recognition with Knowledge Distillation via Multi-Codebook Vector Quantization](https://mlsanity.com/paper/2512.18967/) (2025)
- [TurboBias 2.0: Streaming Context-Biasing for Production-Efficient ASR Systems](https://mlsanity.com/paper/2608.21343/) (2026)
- [PromptKWS: A Novel Prompt-Guided Open-Vocabulary Keyword Spotting Framework](https://mlsanity.com/paper/2608.28640/) (2026)
- [No Detectable Change in Side-Level WER from Prompt-Level Context: A Preregistered Ablation on a Production Oral-History Corpus](https://mlsanity.com/paper/2608.28875/) (2026)

## Abstract (arXiv)

Prompts are crucial to large language models as they provide context information such as topic or logical relationships. Inspired by this, we propose PromptASR, a framework that integrates prompts in end-to-end automatic speech recognition (E2E ASR) systems to achieve contextualized ASR with controllable style of transcriptions. Specifically, a dedicated text encoder encodes the text prompts and the encodings are injected into the speech encoder by cross-attending the features from two modalities. When using the ground truth text from preceding utterances as content prompt, the proposed system achieves 21.9% and 6.8% relative word error rate reductions on a book reading dataset and an in-house dataset compared to a baseline ASR system. The system can also take word-level biasing lists as prompt to improve recognition accuracy on rare words. An additional style prompt can be given to the text encoder and guide the ASR system to output different styles of transcriptions. The code is available at icefall.