# Confidence estimation of classification based on the distribution of the neural network output layer

arXiv: https://arxiv.org/abs/2210.07745 · HTML: https://mlsanity.com/paper/2210.07745/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 14 Oct 2022.

## Builds on (most cited first, 15 of 15)

- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Multiplicative Normalizing Flows for Variational Bayesian Neural Networks](https://mlsanity.com/paper/1703.01961/) (2017)
- [Accelerating Very Deep Convolutional Networks for Classification and Detection](https://mlsanity.com/paper/1505.06798/) (2015)
- [Bayesian Layers: A Module for Neural Network Uncertainty](https://mlsanity.com/paper/1812.03973/) (2018)
- [Adversarial Distillation of Bayesian Neural Network Posteriors](https://mlsanity.com/paper/1806.10317/) (2018)
- [Improving Relation Extraction by Pre-trained Language Representations](https://mlsanity.com/paper/1906.03088/) (2019)
- [Inhibited Softmax for Uncertainty Estimation in Neural Networks](https://mlsanity.com/paper/1810.01861/) (2018)
- [Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction](https://mlsanity.com/paper/2004.08134/) (2020)
- [Noise Contrastive Priors for Functional Uncertainty](https://mlsanity.com/paper/1807.09289/) (2018)
- [Confidence estimation in Deep Neural networks via density modelling](https://mlsanity.com/paper/1707.07013/) (2017)
- [Uncertainty-sensitive Activity Recognition: a Reliability Benchmark and the CARING Models](https://mlsanity.com/paper/2101.00468/) (2021)
- [How the Softmax Output is Misleading for Evaluating the Strength of Adversarial Examples](https://mlsanity.com/paper/1811.08577/) (2018)

## Built on by (most cited first, 8 of 8)

- [AMU-Tuning: Effective Logit Bias for CLIP-based Few-shot Learning](https://mlsanity.com/paper/2404.08958/) (2024)
- [Useful Confidence Measures: Beyond the Max Score](https://mlsanity.com/paper/2210.14070/) (2022)
- [Preventing Shortcut Learning in Medical Image Analysis through Intermediate Layer Knowledge Distillation from Specialist Teachers](https://mlsanity.com/paper/2511.17421/) (2025)
- [Diversifying Deep Ensembles: A Saliency Map Approach for Enhanced OOD Detection, Calibration, and Accuracy](https://mlsanity.com/paper/2305.11616/) (2023)
- [Beyond Graph Model: Reliable VLM Fine-Tuning via Random Graph Adapter](https://mlsanity.com/paper/2507.10355/) (2025)
- [Patchfinder: Leveraging Visual Language Models for Accurate Information Retrieval using Model Uncertainty](https://mlsanity.com/paper/2412.02886/) (2024)
- [Beyond Fixed Thresholds and Domain-Specific Benchmarks for Explainable Multi-Task Classification in Autonomous Vehicles](https://mlsanity.com/paper/2605.04299/) (2026)
- [Imputation-free transformer learning enables robust Alzheimer's disease prediction and calibrated uncertainty quantification across heterogeneous clinical cohorts](https://mlsanity.com/paper/2607.11656/) (2026)

## Abstract (arXiv)

One of the most common problems preventing the application of prediction models in the real world is lack of generalization: The accuracy of models, measured in the benchmark does repeat itself on future data, e.g. in the settings of real business. There is relatively little methods exist that estimate the confidence of prediction models. In this paper, we propose novel methods that, given a neural network classification model, estimate uncertainty of particular predictions generated by this model. Furthermore, we propose a method that, given a model and a confidence level, calculates a threshold that separates prediction generated by this model into two subsets, one of them meets the given confidence level. In contrast to other methods, the proposed methods do not require any changes on existing neural networks, because they simply build on the output logit layer of a common neural network. In particular, the methods infer the confidence of a particular prediction based on the distribution of the logit values corresponding to this prediction. The proposed methods constitute a tool that is recommended for filtering predictions in the process of knowledge extraction, e.g. based on web scrapping, where predictions subsets are identified that maximize the precision on cost of the recall, which is less important due to the availability of data. The method has been tested on different tasks including relation extraction, named entity recognition and image classification to show the significant increase of accuracy achieved.