# Entropy methods for the confidence assessment of probabilistic classification models

arXiv: https://arxiv.org/abs/2103.15157 · HTML: https://mlsanity.com/paper/2103.15157/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 10 references that are in the map.
- First posted on arXiv 28 Mar 2021.

## Builds on (most cited first, 1 of 1)

- [Mitigating Uncertainty in Document Classification](https://mlsanity.com/paper/1907.07590/) (2019)

## Built on by (most cited first, 4 of 4)

- [Fast Entropy-Based Methods of Word-Level Confidence Estimation for End-To-End Automatic Speech Recognition](https://mlsanity.com/paper/2212.08703/) (2022)
- [Self-Aug: Query and Entropy Adaptive Decoding for Large Vision-Language Models](https://mlsanity.com/paper/2510.13315/) (2025)
- [Mask-combine Decoding and Classification Approach for Punctuation Prediction with real-time Inference Constraints](https://mlsanity.com/paper/2112.08098/) (2021)
- [Imputation-free transformer learning enables robust Alzheimer's disease prediction and calibrated uncertainty quantification across heterogeneous clinical cohorts](https://mlsanity.com/paper/2607.11656/) (2026)

## Abstract (arXiv)

Many classification models produce a probability distribution as the outcome of a prediction. This information is generally compressed down to the single class with the highest associated probability. In this paper, we argue that part of the information that is discarded in this process can be in fact used to further evaluate the goodness of models, and in particular the confidence with which each prediction is made. As an application of the ideas presented in this paper, we provide a theoretical explanation of a confidence degradation phenomenon observed in the complement approach to the (Bernoulli) Naive Bayes generative model.