mlsanityData to 7 Oct 2026

Read What You Hear: Reference-Free Hypotheses Evaluation with Acoustic Discrepancy

arXiv 2606.04680 ↗First posted 3 Jun 2026eess.AS cs.CL cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Automatic speech recognition systems commonly rely on reference transcriptions for evaluation, while reference-free approaches often depend on internal confidence estimation or auxiliary language models. We propose READ (Reference-free Hypothesis Evaluation with Acoustic Discrepancy), a novel metric that evaluates ASR hypotheses directly from the speech signal. READ emphasizes the acoustic grounding of hypotheses. It uses a pretrained auto-regressive TTS model to compute the conditional likelihood of speech tokens given a text hypothesis, to measure fine-grained acoustic discrepancy between speech and text. Without additional training, READ can be applied for hypothesis refinement. Experiments show that READ correlates with specific recognition errors and improves ASR outputs, achieving up to 20% relative error rate reduction, with particularly strong gains under noisy conditions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 40 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

27 of 40 references.

References · 27

  1. 2017On Calibration of Modern Neural Networkscited by 4,860
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  4. 2025Qwen2.5-Omni Technical Reportcited by 704
  5. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
  6. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 224
  7. 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 154
  8. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 142
  9. 2023Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognitioncited by 120
  10. 2026Qwen3-ASR Technical Reportcited by 105
  11. 2021SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognitioncited by 77
  12. 2024Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedbackcited by 64
15 more
  1. 2023Can Generative Large Language Models Perform ASR Error Correction?cited by 56
  2. 2020The ASRU 2019 Mandarin-English Code-Switching Speech Recognition Challenge: Open Datasets, Tracks, Methods and Resultscited by 43
  3. 2023HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Modelscited by 41
  4. 2021Semantic Distance: A New Metric for ASR Performance Analysis Towards Spoken Language Understandingcited by 40
  5. 2024Large Language Models are Efficient Learners of Noise-Robust Speech Recognitioncited by 30
  6. 2025IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech Systemcited by 27
  7. 2021An evaluation of word-level confidence estimation for end-to-end automatic speech recognitioncited by 18
  8. 2022TALCS: An Open-Source Mandarin-English Code-Switching Corpus and a Speech Recognition Baselinecited by 17
  9. 2021ASR Rescoring and Confidence Estimation with ELECTRAcited by 16
  10. 2024Listen Again and Choose the Right Answer: A New Paradigm for Automatic Speech Recognition with Large Language Modelscited by 13
  11. 2020Word Error Rate Estimation Without ASR Output: e-WER2cited by 8
  12. 2024Attention-Constrained Inference for Robust Decoder-Only Text-to-Speechcited by 6
  13. 2024ProGRes: Prompted Generative Rescoring on ASR n-Bestcited by 5
  14. 2023Fast Word Error Rate Estimation Using Self-Supervised Representations for Speech and Textcited by 4
  15. 2025On the Robust Approximation of ASR Metricscited by 2

Built on by

Not cited yet.