Whisper-CD: Accurate Long-Form Speech Recognition using Multi-Negative Contrastive Decoding
Not on the map. Why?
Abstract (arXiv)
Long-form speech recognition with large encoder-decoder models such as Whisper often exhibit hallucinations, repetition loops, and content omissions. These errors can accumulate and be further amplified when the previous segment's transcription is used as decoding context. We propose Whisper-CD, a training-free contrastive decoding framework that contrasts clean-audio logits against negative logits computed from three acoustically motivated perturbations: Gaussian noise injection, silence signal, and audio temporal shift. We aggregate these negatives via the log-sum-exp operator, building a unified multi-negative objective for token-by-token decoding. Across five English long-form benchmarks, Whisper-CD reduces WER by up to 24.3pp on CORAAL and shows 48% faster token generation throughput than beam search. Because Whisper-CD operates purely at inference time, it can be applied as a drop-in replacement to already-deployed Whisper systems without retraining.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 23 of its 28 references that are in the map. First posted on arXiv 6 Mar 2026.
Builds on
23 of 28 references.
References · 23
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
- 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
- 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
- 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
- 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
- 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 224
- 2026Qwen3-ASR Technical Reportcited by 105
- 2023Generative Speech Recognition Error Correction with Large Language Models and Task-Activating Promptingcited by 71
- 2024Careless Whisper: Speech-to-Text Hallucination Harmscited by 53
- 2022Earnings-22: A Practical Benchmark for Accents in the Wildcited by 46
- 2023Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decodingcited by 42
11 more
- 2025Voxtralcited by 41
- 2024CrisperWhisper: Accurate Timestamps on Verbatim Speech Transcriptionscited by 33
- 2025Investigation of Whisper ASR Hallucinations Induced by Non-Speech Audiocited by 31
- 2022Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognitioncited by 19
- 2022E2E Segmenter: Joint Segmenting and Decoding for Long-Form ASRcited by 17
- 2025Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Modelscited by 17
- 2024Listen Again and Choose the Right Answer: A New Paradigm for Automatic Speech Recognition with Large Language Modelscited by 13
- 2023Updated Corpora and Benchmarks for Long-Form Speech Recognitioncited by 10
- 2024Efficient Streaming LLM for Speech Recognitioncited by 10
- 2022LongFNT: Long-form Speech Recognition with Factorized Neural Transducercited by 6
- 2025ChunkFormer: Masked Chunking Conformer For Long-Form Speech Transcriptioncited by 3
Built on by
Citing papers per year: 2026: 3.
3 papers.