# Fast Entropy-Based Methods of Word-Level Confidence Estimation for End-To-End Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2212.08703 · HTML: https://mlsanity.com/paper/2212.08703/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 41 references that are in the map.
- First posted on arXiv 16 Dec 2022.

## Builds on (most cited first, 17 of 17)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [NeMo: a toolkit for building AI applications using Neural Modules](https://mlsanity.com/paper/1909.09577/) (2019)
- [Improved Noisy Student Training for Automatic Speech Recognition](https://mlsanity.com/paper/2005.09629/) (2020)
- [SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification](https://mlsanity.com/paper/2105.11043/) (2021)
- [Confidence Estimation for Attention-based Sequence-to-sequence Models for Speech Recognition](https://mlsanity.com/paper/2010.11428/) (2020)
- [Learning Word-Level Confidence For Subword End-to-End ASR](https://mlsanity.com/paper/2103.06716/) (2021)
- [An evaluation of word-level confidence estimation for end-to-end automatic speech recognition](https://mlsanity.com/paper/2101.05525/) (2021)
- [Active and Semi-Supervised Learning in ASR: Benefits on the Acoustic and Language Models](https://mlsanity.com/paper/1903.02852/) (2019)
- [Residual Energy-Based Models for End-to-End Speech Recognition](https://mlsanity.com/paper/2103.14152/) (2021)
- [Multi-Task Learning for End-to-End ASR Word and Utterance Confidence with Deletion Prediction](https://mlsanity.com/paper/2104.12870/) (2021)
- [On Modeling ASR Word Confidence](https://mlsanity.com/paper/1907.09636/) (2019)
- [Entropy methods for the confidence assessment of probabilistic classification models](https://mlsanity.com/paper/2103.15157/) (2021)
- [Word-level confidence estimation for RNN transducers](https://mlsanity.com/paper/2110.15222/) (2021)

## Built on by (most cited first, 7 of 7)

- [Empowering Low-Resource Language ASR via Large-Scale Pseudo Labeling](https://mlsanity.com/paper/2408.14026/) (2024)
- [Confidence-based Ensembles of End-to-End Speech Recognition Models](https://mlsanity.com/paper/2306.15824/) (2023)
- [BaFTA: Backprop-Free Test-Time Adaptation For Zero-Shot Vision-Language Models](https://mlsanity.com/paper/2406.11309/) (2024)
- [Adopting Whisper for Confidence Estimation](https://mlsanity.com/paper/2502.13446/) (2025)
- [TeLeS: Temporal Lexeme Similarity Score to Estimate Confidence in End-to-End ASR](https://mlsanity.com/paper/2401.03251/) (2024)
- [Confidence-Guided Error Correction for Disordered Speech Recognition](https://mlsanity.com/paper/2509.25048/) (2025)
- [Enhancing ASR Performance in the Medical Domain for Dravidian Languages](https://mlsanity.com/paper/2604.19797/) (2026)

## Abstract (arXiv)

This paper presents a class of new fast non-trainable entropy-based confidence estimation methods for automatic speech recognition. We show how per-frame entropy values can be normalized and aggregated to obtain a confidence measure per unit and per word for Connectionist Temporal Classification (CTC) and Recurrent Neural Network Transducer (RNN-T) models. Proposed methods have similar computational complexity to the traditional method based on the maximum per-frame probability, but they are more adjustable, have a wider effective threshold range, and better push apart the confidence distributions of correct and incorrect words. We evaluate the proposed confidence measures on LibriSpeech test sets, and show that they are up to 2 and 4 times better than confidence estimation based on the maximum per-frame probability at detecting incorrect words for Conformer-CTC and Conformer-RNN-T models, respectively.