# Energy-Based Open-World Uncertainty Modeling for Confidence Calibration

arXiv: https://arxiv.org/abs/2107.12628 · HTML: https://mlsanity.com/paper/2107.12628/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 42 references that are in the map.
- First posted on arXiv 27 Jul 2021.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data](https://mlsanity.com/paper/2002.11297/) (2020)
- [Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One](https://mlsanity.com/paper/1912.03263/) (2019)
- [Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples](https://mlsanity.com/paper/1711.09325/) (2017)
- [Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem](https://mlsanity.com/paper/1812.05720/) (2018)
- [On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks](https://mlsanity.com/paper/1905.11001/) (2019)
- [To Trust Or Not To Trust A Classifier](https://mlsanity.com/paper/1805.11783/) (2018)
- [Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model](https://mlsanity.com/paper/1904.09770/) (2019)
- [On Calibration and Out-of-domain Generalization](https://mlsanity.com/paper/2102.10395/) (2021)
- [Learning Energy-Based Models by Diffusion Recovery Likelihood](https://mlsanity.com/paper/2012.08125/) (2020)
- [Energy-based Generative Adversarial Network](https://mlsanity.com/paper/1609.03126/) (2016)
- [Generalized Energy Based Models](https://mlsanity.com/paper/2003.05033/) (2020)
- [Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling](https://mlsanity.com/paper/2003.06060/) (2020)
- [No MCMC for me: Amortized sampling for fast and stable training of energy-based models](https://mlsanity.com/paper/2010.04230/) (2020)
- [Deep Directed Generative Models with Energy-Based Probability Estimation](https://mlsanity.com/paper/1606.03439/) (2016)
- [Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks](https://mlsanity.com/paper/2007.05134/) (2020)
- [Distance-Based Learning from Errors for Confidence Calibration](https://mlsanity.com/paper/1912.01730/) (2019)

## Built on by (most cited first, 30 of 46)

- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [OpenOOD: Benchmarking Generalized Out-of-Distribution Detection](https://mlsanity.com/paper/2210.07242/) (2022)
- [Negative Label Guided OOD Detection with Pretrained Vision-Language Models](https://mlsanity.com/paper/2403.20078/) (2024)
- [Learning with Mixture of Prototypes for Out-of-Distribution Detection](https://mlsanity.com/paper/2402.02653/) (2024)
- [Energy-based Latent Aligner for Incremental Learning](https://mlsanity.com/paper/2203.14952/) (2022)
- [Open-world Semantic Segmentation for LIDAR Point Clouds](https://mlsanity.com/paper/2207.01452/) (2022)
- [Watermarking for Out-of-distribution Detection](https://mlsanity.com/paper/2210.15198/) (2022)
- [On Calibrating Semantic Segmentation Models: Analyses and An Algorithm](https://mlsanity.com/paper/2212.12053/) (2022)
- [OpenTAL: Towards Open Set Temporal Action Localization](https://mlsanity.com/paper/2203.05114/) (2022)
- [Out-of-Distribution Detection with Semantic Mismatch under Masking](https://mlsanity.com/paper/2208.00446/) (2022)
- [LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Models](https://mlsanity.com/paper/2407.08966/) (2024)
- [AdaNeg: Adaptive Negative Proxy Guided OOD Detection with Vision-Language Models](https://mlsanity.com/paper/2410.20149/) (2024)
- [Uncertainty Quantification for Machine Learning in Healthcare: A Survey](https://mlsanity.com/paper/2505.02874/) (2025)
- [STAMP: Outlier-Aware Test-Time Adaptation with Stable Memory Replay](https://mlsanity.com/paper/2407.15773/) (2024)
- [Out-Of-Distribution Detection In Unsupervised Continual Learning](https://mlsanity.com/paper/2204.05462/) (2022)
- [Your Autoregressive Generative Model Can be Better If You Treat It as an Energy-Based One](https://mlsanity.com/paper/2206.12840/) (2022)
- [Calibrating the Rigged Lottery: Making All Tickets Reliable](https://mlsanity.com/paper/2302.09369/) (2023)
- [Morse Neural Networks for Uncertainty Quantification](https://mlsanity.com/paper/2307.00667/) (2023)
- [PDF: A Probability-Driven Framework for Open World 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2404.00979/) (2024)
- [Improving Out-of-Distribution Detection via Dynamic Covariance Calibration](https://mlsanity.com/paper/2506.09399/) (2025)
- [Reconstruction guided Meta-learning for Few Shot Open Set Recognition](https://mlsanity.com/paper/2108.00340/) (2021)
- [Probabilistic Approach for Road-Users Detection](https://mlsanity.com/paper/2112.01360/) (2021)
- [Learning Large Margin Sparse Embeddings for Open Set Medical Diagnosis](https://mlsanity.com/paper/2307.04541/) (2023)
- [Open-Set Facial Expression Recognition](https://mlsanity.com/paper/2401.12507/) (2024)
- [BECAUSE: Bilinear Causal Representation for Generalizable Offline Model-based Reinforcement Learning](https://mlsanity.com/paper/2407.10967/) (2024)
- [Latent Space Energy-based Model for Fine-grained Open Set Recognition](https://mlsanity.com/paper/2309.10711/) (2023)
- [Entropy Heat-Mapping: Localizing GPT-Based OCR Errors with Sliding-Window Shannon Analysis](https://mlsanity.com/paper/2505.00746/) (2025)
- [h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objective](https://mlsanity.com/paper/2506.17968/) (2025)
- [A Retrospect to Multi-prompt Learning across Vision and Language](https://mlsanity.com/paper/2511.00191/) (2025)
- [Cross-modal Proxy Evolving for OOD Detection with Vision-Language Models](https://mlsanity.com/paper/2601.08476/) (2026)

## Abstract (arXiv)

Confidence calibration is of great importance to the reliability of decisions made by machine learning systems. However, discriminative classifiers based on deep neural networks are often criticized for producing overconfident predictions that fail to reflect the true correctness likelihood of classification accuracy. We argue that such an inability to model uncertainty is mainly caused by the closed-world nature in softmax: a model trained by the cross-entropy loss will be forced to classify input into one of K pre-defined categories with high probability. To address this problem, we for the first time propose a novel K+1-way softmax formulation, which incorporates the modeling of open-world uncertainty as the extra dimension. To unify the learning of the original K-way classification task and the extra dimension that models uncertainty, we propose a novel energy-based objective function, and moreover, theoretically prove that optimizing such an objective essentially forces the extra dimension to capture the marginal data distribution. Extensive experiments show that our approach, Energy-based Open-World Softmax (EOW-Softmax), is superior to existing state-of-the-art methods in improving confidence calibration.