# Identifying and Calibrating Overconfidence in Noisy Speech Recognition

arXiv: https://arxiv.org/abs/2509.07195 · HTML: https://mlsanity.com/paper/2509.07195/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Sep 2025.

## Built on by (most cited first, 2 of 2)

- [RAS: a Reliability Oriented Metric for Automatic Speech Recognition](https://mlsanity.com/paper/2604.24278/) (2026)
- [Attention-Guided Reliability Scaling for Contrastive Decoding in Robust Audio-Visual Speech Recognition](https://mlsanity.com/paper/2608.26213/) (2026)

## Abstract (arXiv)

Modern end-to-end automatic speech recognition (ASR) models like Whisper not only suffer from reduced recognition accuracy in noise, but also exhibit overconfidence - assigning high confidence to wrong predictions. We conduct a systematic analysis of Whisper's behavior in additive noise conditions and find that overconfident errors increase dramatically at low signal-to-noise ratios, with 10-20% of tokens incorrectly predicted with confidence above 0.7. To mitigate this, we propose a lightweight, post-hoc calibration framework that detects potential overconfidence and applies temperature scaling selectively to those tokens, without altering the underlying ASR model. Evaluations on the R-SPIN dataset demonstrate that, in the low signal-to-noise ratio range (-18 to -5 dB), our method reduces the expected calibration error (ECE) by 58% and triples the normalized cross entropy (NCE), yielding more reliable confidence estimates under severe noise conditions.