mlsanityData to 13 Oct 2026

Can Contextual Biasing Remain Effective with Whisper and GPT-2?

arXiv 2306.01942 ↗First posted 2 Jun 2023cs.CL cs.SD eess.AScited by 21

Not on the map. Why?

Abstract (arXiv)

End-to-end automatic speech recognition (ASR) and large language models, such as Whisper and GPT-2, have recently been scaled to use vast amounts of training data. Despite the large amount of training data, infrequent content words that occur in a particular task may still exhibit poor ASR performance, with contextual biasing a possible remedy. This paper investigates the effectiveness of neural contextual biasing for Whisper combined with GPT-2. Specifically, this paper proposes integrating an adapted tree-constrained pointer generator (TCPGen) component for Whisper and a dedicated training scheme to dynamically adjust the final output without modifying any Whisper model parameters. Experiments across three datasets show a considerable reduction in errors on biasing words with a biasing list of 1000 words. Contextual biasing was more effective when applied to domain-specific data and can boost the performance of Whisper and GPT-2 without losing their generality.

Key facts. Cited by 21 papers in the map (data through 13 Oct 2026). Builds on 19 of its 25 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

19 of 25 references.

References · 19

  1. 2017Attention Is All You Needcited by 60,106
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,004
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,870
  4. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,288
  5. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 229
  6. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 197
  7. 2018Deep context: end-to-end contextual speech recognitioncited by 137
  8. 2021Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusioncited by 100
  9. 2020Contextual RNN-T For Open Domain ASRcited by 85
  10. 2020Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognitioncited by 80
  11. 2020Deep Shallow Fusion for RNN-T Personalizationcited by 64
  12. 2021Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognitioncited by 32
7 more
  1. 2021Instant One-Shot Word-Learning for Context-Specific Neural Sequence-to-Sequence Speech Recognitioncited by 30
  2. 2018Contextual Speech Recognition with Difficult Negative Training Examplescited by 28
  3. 2018End-to-end contextual speech recognition using class language models and a token passing decodercited by 26
  4. 2022Towards Contextual Spelling Correction for Customization of End-to-end Speech Recognition Systemscited by 17
  5. 2022Tree-constrained Pointer Generator with Graph Neural Network Encodings for Contextual Speech Recognitioncited by 17
  6. 2021Adapting GPT, GPT-2 and BERT Language Models for Speech Recognitioncited by 16
  7. 2022End-to-end Spoken Language Understanding with Tree-constrained Pointer Generatorcited by 6

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 3; 2024: 7; 2025: 6; 2026: 5.

21 papers.

Sort:

By area:

  1. 2023SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translationcited by 75
  2. 2023Phoneme-aware Encoding for Prefix-tree-based Contextual ASRcited by 13
  3. 2024Contextualized End-to-end Automatic Speech Recognition with Intermediate Biasing Losscited by 9
  4. 2024Improving Domain-Specific ASR with LLM-Generated Contextual Descriptionscited by 6
  5. 2024Keyword-Guided Adaptation of Automatic Speech Recognitioncited by 5
  6. 2024Enhanced ASR Robustness to Packet Loss with a Front-End Adaptation Networkcited by 4
  7. 2024SMILE: Speech Meta In-Context Learning for Low-Resource Language Automatic Speech Recognitioncited by 4
  8. 2025OWSM-Biasing: Contextualizing Open Whisper-Style Speech Models for Automatic Speech Recognition with Dynamic Vocabularycited by 4
  9. 2024Improving Whisper's Recognition Performance for Under-Represented Language Kazakh Leveraging Unpaired Speech and Textcited by 3
  10. 2025Zero-shot Context Biasing with Trie-based Decoding using Synthetic Multi-Pronunciationcited by 3
  11. 2026IndicContextEval: A Benchmark for Evaluating Context Utilisation in Audio Large Language Models Across 8 Indic Languagescited by 3
  12. 2023Enhancing Quantised End-to-End ASR Models via Personalisationcited by 1
9 more
  1. 2025Lightweight Prompt Biasing for Contextualized End-to-End ASR Systemscited by 1
  2. 2025Improving Contextual ASR via Multi-grained Fusion with Large Language Modelscited by 1
  3. 2025Improving Synthetic Data Training for Contextual Biasing Models with a Keyword-Aware Cost Functioncited by 1
  4. 2024Mai Ho'om=auna i ka 'Ai: Language Models Improve Automatic Speech Recognition in Hawaiiancited by 0
  5. 2025Efficient Trie-based Biasing using K-step Prediction for Rare Word Recognitioncited by 0
  6. 2026ASTRA: A Scalable Next-Generation ATCO Training Simulator with Autonomous Simpilotscited by 0
  7. 2026No Detectable Change in Side-Level WER from Prompt-Level Context: A Preregistered Ablation on a Production Oral-History Corpuscited by 0
  8. 2026Anchoring Speech with Semantics: A Multimodal Adapter Mechanism for Automatic Speech Recognition in Low-Resource Languagescited by 0
  9. 2026Talk2Agent: Benchmarking Voice Interfaces for Text Agentscited by 0