mlsanityData to 14 Oct 2026

CTC-Assisted LLM-Based Contextual ASR

arXiv 2411.06437 ↗First posted 10 Nov 2024eess.AS cs.AI cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

Contextual ASR or hotword customization holds substantial practical value. Despite the impressive performance of current end-to-end (E2E) automatic speech recognition (ASR) systems, they often face challenges in accurately recognizing rare words. Typical E2E contextual ASR models commonly feature complex architectures and decoding mechanisms, limited in performance and susceptible to interference from distractor words. With large language model (LLM)-based ASR models emerging as the new mainstream, we propose a CTC-Assisted LLM-Based Contextual ASR model with an efficient filtering algorithm. By using coarse CTC decoding results to filter potential relevant hotwords and incorporating them into LLM prompt input, our model attains WER/B-WER of 1.27%/3.67% and 2.72%/8.02% on the Librispeech test-clean and test-other sets targeting on recognizing rare long-tail words, demonstrating significant improvements compared to the baseline LLM-based ASR model, and substantially surpassing other related work. More remarkably, with the help of the large language model and proposed filtering algorithm, our contextual ASR model still performs well with 2000 biasing words.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 17 of its 25 references that are in the map. First posted on arXiv 10 Nov 2024.

Builds on

17 of 25 references.

References · 17

  1. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  2. 2023On decoder-only architecture for speech-to-text and large language model integrationcited by 145
  3. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 140
  4. 2018Deep context: end-to-end contextual speech recognitioncited by 137
  5. 2021Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusioncited by 101
  6. 2024An Embarrassingly Simple Approach for LLM with Strong ASR Capacitycited by 88
  7. 2020Contextual RNN-T For Open Domain ASRcited by 85
  8. 2020Class LM and word mapping for contextual biasing in End-to-End ASRcited by 42
  9. 2023Contextualized End-to-End Speech Recognition with Contextual Phrase Prediction Networkcited by 31
  10. 2018End-to-end contextual speech recognition using class language models and a token passing decodercited by 26
  11. 2020CIF-based Collaborative Decoding for End-to-end Contextual Speech Recognitioncited by 21
  12. 2024Contextualized Automatic Speech Recognition with Attention-Based Bias Phrase Boosted Beam Searchcited by 19
5 more
  1. 2023SeACo-Paraformer: A Non-Autoregressive ASR System with Flexible and Effective Hotword Customization Abilitycited by 18
  2. 2022Tree-constrained Pointer Generator with Graph Neural Network Encodings for Contextual Speech Recognitioncited by 17
  3. 2024Improving ASR Contextual Biasing with Guided Attentioncited by 17
  4. 2023Phoneme-aware Encoding for Prefix-tree-based Contextual ASRcited by 13
  5. 2023Two Stage Contextual Word Filtering for Context bias in Unified Streaming and Non-streaming Transducercited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 9.

19 papers.

Sort:

By area:

  1. 2025Audio-CoT: Exploring Chain-of-Thought Reasoning in Large Audio Language Modelcited by 62
  2. 2025Evaluation of LLMs in Speech is Often Flawed: Test Set Contamination in Large Language Models for Speech Recognitioncited by 9
  3. 2026SLAM-LLM: A Modular, Open-Source Multimodal Large Language Model Framework and Best Practice for Speech, Language, Audio and Music Processingcited by 8
  4. 2025Zero-shot Context Biasing with Trie-based Decoding using Synthetic Multi-Pronunciationcited by 3
  5. 2025Boosting CTC-Based ASR Using LLM-Based Intermediate Loss Regularizationcited by 2
  6. 2025Lightweight Prompt Biasing for Contextualized End-to-End ASR Systemscited by 1
  7. 2025CMT-LLM: Contextual Multi-Talker ASR Utilizing Large Language Modelscited by 1
  8. 2025Enhancing the Robustness of Contextual ASR to Varying Biasing Information Volumes Through Purified Semantic Correlation Joint Modelingcited by 1
  9. 2025PAC: Pronunciation-Aware Contextualized Large Language Model-based Automatic Speech Recognitioncited by 1
  10. 2026When Helpful Context Leaks: Privacy Risks in Domain-Adapted ASRcited by 1
  11. 2026MEUSLI: a Multilingual Projector for LLM-based ASR and Beyondcited by 1
  12. 2025Understanding Zero-shot Rare Word Recognition Improvements Through LLM Integrationcited by 0
7 more
  1. 2025H-PRM: A Pluggable Hotword Pre-Retrieval Module for Various Speech Recognition Systemscited by 0
  2. 2026RASST: Retrieval-Augmented Simultaneous Speech Translationcited by 0
  3. 2026CALM: Joint Contextual Acoustic-Linguistic Modeling for Personalization of Multi-Speaker ASRcited by 0
  4. 2026Speak in Context: Multilingual ASR with Speech Context Alignment via Contrastive Learningcited by 0
  5. 2026COALA: Robust Contextualized Speech-augmented Language Modeling for ASR via Contrastive Regularizer and Biasing Score Estimationcited by 0
  6. 2026PTC-Bias: Phoneme-Level Temporal Competition for Bias Retrieval and Post-Decoding Correction in Speech LLMscited by 0
  7. 2026LLM-HB: Language-Aware LLM-Guided Hotword Biasing for Code-Switching ASRcited by 0