mlsanityData to 14 Oct 2026

PromptASR for contextualized ASR with controllable style

arXiv 2309.07414 ↗First posted 14 Sep 2023eess.AS cs.CL cs.SDcited by 19

Not on the map. Why?

Abstract (arXiv)

Prompts are crucial to large language models as they provide context information such as topic or logical relationships. Inspired by this, we propose PromptASR, a framework that integrates prompts in end-to-end automatic speech recognition (E2E ASR) systems to achieve contextualized ASR with controllable style of transcriptions. Specifically, a dedicated text encoder encodes the text prompts and the encodings are injected into the speech encoder by cross-attending the features from two modalities. When using the ground truth text from preceding utterances as content prompt, the proposed system achieves 21.9% and 6.8% relative word error rate reductions on a book reading dataset and an in-house dataset compared to a baseline ASR system. The system can also take word-level biasing lists as prompt to improve recognition accuracy on rare words. An additional style prompt can be given to the text encoder and guide the ASR system to output different styles of transcriptions. The code is available at icefall.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 18 of its 24 references that are in the map. First posted on arXiv 14 Sep 2023.

Builds on

18 of 24 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  4. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  6. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  7. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  8. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  9. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 425
  10. 2017An analysis of incorporating an external language model into a sequence-to-sequence modelcited by 111
  11. 2021Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusioncited by 101
  12. 2019Recognizing long-form speech using streaming end-to-end modelscited by 77
6 more
  1. 2023Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognitioncited by 48
  2. 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
  3. 2021Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognitioncited by 32
  4. 2023Contextualized End-to-End Speech Recognition with Contextual Phrase Prediction Networkcited by 31
  5. 2021Adapting GPT, GPT-2 and BERT Language Models for Speech Recognitioncited by 16
  6. 2021Attentive Contextual Carryover for Multi-Turn End-to-End Spoken Language Understandingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 10; 2026: 4.

19 papers.

Sort:

By area:

  1. 2024Chain-of-Thought Prompting for Speech Translationcited by 17
  2. 2024Improving Neural Biasing for Contextual Speech Recognition by Early Context Injection and Text Perturbationcited by 11
  3. 2024Fast Context-Biasing for CTC and Transducer ASR models with CTC-based Word Spottercited by 7
  4. 2024Improving Domain-Specific ASR with LLM-Generated Contextual Descriptionscited by 6
  5. 2025OWSM-Biasing: Contextualizing Open Whisper-Style Speech Models for Automatic Speech Recognition with Dynamic Vocabularycited by 4
  6. 2025TurboBias: Universal ASR Context-Biasing powered by GPU-accelerated Phrase-Boosting Treecited by 4
  7. 2024Optimizing Contextual Speech Recognition Using Vector Quantization for Efficient Retrievalcited by 3
  8. 2025PROFASR-BENCH: A Benchmark for Context-Conditioned ASR in High-Stakes Professional Speechcited by 3
  9. 2025Lightweight Prompt Biasing for Contextualized End-to-End ASR Systemscited by 1
  10. 2025Enhancing the Robustness of Contextual ASR to Varying Biasing Information Volumes Through Purified Semantic Correlation Joint Modelingcited by 1
  11. 2026How to Recognize New Words: A Comparison Between Context Biasing Methods and Speech LLMscited by 1
  12. 2025Bi-directional Context-Enhanced Speech Large Language Models for Multilingual Conversational ASRcited by 0
7 more
  1. 2025Context Biasing for Pronunciation-Orthography Mismatch in Automatic Speech Recognitioncited by 0
  2. 2025Attention2Probability: Attention-Driven Terminology Probability Estimation for Robust Speech-to-Text Systemcited by 0
  3. 2025TokenVerse++: Towards Flexible Multitask Learning with Dynamic Task Activationcited by 0
  4. 2025Enhancing Fully Formatted End-to-End Speech Recognition with Knowledge Distillation via Multi-Codebook Vector Quantizationcited by 0
  5. 2026TurboBias 2.0: Streaming Context-Biasing for Production-Efficient ASR Systemscited by 0
  6. 2026PromptKWS: A Novel Prompt-Guided Open-Vocabulary Keyword Spotting Frameworkcited by 0
  7. 2026No Detectable Change in Side-Level WER from Prompt-Level Context: A Preregistered Ablation on a Production Oral-History Corpuscited by 0