mlsanityData to 19 Oct 2026

Towards Deep Contextual Reasoning from Broad Descriptions for ASR with Speech-LLM via Metadata-Driven Reasoning Chains

arXiv 2606.10838 ↗First posted 9 Jun 2026eess.AScited by 0

On the map: Speech and audio language models → Speech recognition models and training

Abstract (arXiv)

Speech recognition often fails on rare, domain-specific terms and context-related named entities. Existing contextualization techniques typically bias decoding with keywords or phrase lists, which does not scale well or exploit deeper knowledge. We propose a training method that teaches a speech-LLM to use broad descriptions (e.g. from videos) as weak semantic priors to perform contextual reasoning grounded in the audio. We build 400 hours of reasoning-augmented speech data by pairing erroneous hypotheses with video metadata and LLM-generated reasoning explanations that justify context-driven corrections. We finetune the speech-LLM to perform chain-of-thought reasoning: generate an initial transcript, then reason over the context, and finally return a corrected transcript. On held-out YouTube-derived test sets, our approach reduces errors, with specific improvements on rare words and named entities, and lays groundwork for deeper contextual reasoning in speech recognition.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Speech recognition models and training, one of 11 directions in Speech and audio language models. Builds on 29 of its 33 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

29 of 33 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Phonetically-Augmented Discriminative Rescoring for Voice Search Error Correction6 Jun
  2. 2025BR-ASR: Efficient and Scalable Bias Retrieval Framework for Contextual Biasing ASR in Speech LLM25 May
  3. 2025Predicting Compact Phrasal Rewrites with Large Language Models for ASR Post Editing23 Jan
  4. 2024Speech Recognition Rescoring with Large Speech-Text Foundation Models25 Sep
  5. 2024SALSA: Speedy ASR-LLM Synchronous Aggregation29 Aug
  6. 2024Towards interfacing large language models with ASR systems using confidence measures and prompting31 Jul
  7. 2024Improving Domain-Specific ASR with LLM-Generated Contextual Descriptions25 Jul
  8. 2024Contextualized End-to-end Automatic Speech Recognition with Intermediate Biasing Loss23 Jun
  9. 2024Listen Again and Choose the Right Answer: A New Paradigm for Automatic Speech Recognition with Large Language Models16 May
  10. 2023Contextual Biasing of Named-Entities with Large Language Models1 Sep
  11. 2022Robust Speech Recognition via Large-Scale Weak Supervision6 Dec

Other references · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,923
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,627
  3. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,468
  4. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 755
  5. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 497
  6. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 349
  7. 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilitiescited by 187
  8. 2025Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Modelscited by 118
  9. 2025Audio-CoT: Exploring Chain-of-Thought Reasoning in Large Audio Language Modelcited by 80
  10. 2025DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignmentcited by 62
  11. 2024DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Datacited by 53
  12. 2025SAKURA: On the Multi-hop Reasoning of Large Audio-Language Models Based on Speech and Audio Informationcited by 47
6 more
  1. 2025OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaningcited by 39
  2. 2024DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignmentcited by 38
  3. 2024Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoningcited by 36
  4. 2023SlideSpeech: A Large-Scale Slide-Enriched Audio-Visual Corpuscited by 26
  5. 2024SlideAVSR: A Dataset of Paper Explanation Videos for Audio-Visual Speech Recognitioncited by 10
  6. 2024M3AV: A Multimodal, Multigenre, and Multipurpose Audio-Visual Academic Lecture Datasetcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Speech LLMs are Contextual Reasoning Transcribers
  2. 2026LaSR: Context-Aware Speech Recognition via Latent Reasoning
  3. 2026Multimodal Conversational Context for LLM-Based ASR: Data Construction, Training, and Benchmark
  4. 2026Think Before You Transcribe: Context-Steered Reasoning for Hotword-Aware Speech Recognition
  5. 2026Reasoning LLM Improves Speaker Recognition in Long-form TV Dramas
  6. 2025Whispering Context: Distilling Syntax and Semantics for Long Speech Transcripts
  7. 2023End-to-End Speech Recognition Contextualization with Large Language Models
  8. 2026ALARM: Audio-Language Alignment for Reasoning Models