mlsanityData to 14 Oct 2026

Evaluating ASR Confidence Scores for Automated Error Detection in User-Assisted Correction Interfaces

arXiv 2503.15124 ↗First posted 19 Mar 2025cs.HC cs.CL cs.SDcited by 6

Not on the map. Why?

Abstract (arXiv)

Despite advances in Automatic Speech Recognition (ASR), transcription errors persist and require manual correction. Confidence scores, which indicate the certainty of ASR results, could assist users in identifying and correcting errors. This study evaluates the reliability of confidence scores for error detection through a comprehensive analysis of end-to-end ASR models and a user study with 36 participants. The results show that while confidence scores correlate with transcription accuracy, their error detection performance is limited. Classifiers frequently miss errors or generate many false positives, undermining their practical utility. Confidence-based error detection neither improved correction efficiency nor was perceived as helpful by participants. These findings highlight the limitations of confidence scores and the need for more sophisticated approaches to improve user interaction and explainability of ASR results.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 19 of its 69 references that are in the map. First posted on arXiv 19 Mar 2025.

Builds on

19 of 69 references.

References · 19

  1. 2017On Calibration of Modern Neural Networkscited by 4,960
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  5. 2015Listen, Attend and Spellcited by 543
  6. 2015Attention-Based Models for Speech Recognitioncited by 401
  7. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  8. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 257
  9. 2015Semi-Supervised Learning with Ladder Networkscited by 247
  10. 2018Learning Confidence for Out-of-Distribution Detection in Neural Networkscited by 245
  11. 2020Improved Noisy Student Training for Automatic Speech Recognitioncited by 141
  12. 2022Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understandingcited by 91
7 more
  1. 2020Confidence Estimation for Attention-based Sequence-to-sequence Models for Speech Recognitioncited by 35
  2. 2016Active Learning for Speech Recognition: the Power of Gradientscited by 25
  3. 2024Measuring the Accuracy of Automatic Speech Recognition Solutionscited by 15
  4. 2018Bi-Directional Lattice Recurrent Neural Networks for Confidence Estimationcited by 11
  5. 2021Multi-Task Learning for End-to-End ASR Word and Utterance Confidence with Deletion Predictioncited by 8
  6. 2018A Corpus for Modeling Word Importance in Spoken Dialogue Transcriptscited by 4
  7. 2021Social, Environmental, and Technical: Factors at Play in the Current Use and Future Design of Small-Group Captioningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2025Data-Efficient ASR Personalization for Non-Normative Speech Using an Uncertainty-Based Phoneme Difficulty Score for Guided Samplingcited by 1
  2. 2026When Misinformation Speaks and Converses: Rethinking Fact-Checking in Audio Platformscited by 1
  3. 2026Demonstration of Adapt4Me: An Uncertainty-Aware Authoring Environment for Personalizing Automatic Speech Recognition to Non-normative Speechcited by 0
  4. 2026Utterance-Level Methods for Identifying Reliable ASR-Output for Child Speechcited by 0
  5. 2026Proactive for Uncertainty: Cause-Aware Error Diagnosis and Interactive Clarification for Spoken Dialogue Systemscited by 0
  6. 2026Transcribing Children's Speech: ASR Performance and Obtaining Reliable Orthographic Transcriptionscited by 0