# Confidence Estimation Using Unlabeled Data

arXiv: https://arxiv.org/abs/2307.10440 · HTML: https://mlsanity.com/paper/2307.10440/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 45 references that are in the map.
- First posted on arXiv 19 Jul 2023.

## Builds on (most cited first, 30 of 30)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1710.05006/) (2017)
- [Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision](https://mlsanity.com/paper/2106.01226/) (2021)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [S4L: Self-Supervised Semi-Supervised Learning](https://mlsanity.com/paper/1905.03670/) (2019)
- [End-to-End Semi-Supervised Object Detection with Soft Teacher](https://mlsanity.com/paper/2106.09018/) (2021)
- [Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels](https://mlsanity.com/paper/2203.03884/) (2022)
- [Mix-n-Match: Ensemble and Compositional Methods for Uncertainty Calibration in Deep Learning](https://mlsanity.com/paper/2003.07329/) (2020)
- [Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Cost](https://mlsanity.com/paper/1910.07153/) (2019)
- [Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers](https://mlsanity.com/paper/1805.08206/) (2018)
- [Confidence-Aware Learning for Deep Neural Networks](https://mlsanity.com/paper/2007.01458/) (2020)
- [ViewAL: Active Learning with Viewpoint Entropy for Semantic Segmentation](https://mlsanity.com/paper/1911.11789/) (2019)
- [Assessing Reliability and Challenges of Uncertainty Estimations for Medical Image Segmentation](https://mlsanity.com/paper/1907.03338/) (2019)
- [Semi-Supervised Active Learning with Temporal Output Discrepancy](https://mlsanity.com/paper/2107.14153/) (2021)
- [Distance-based Confidence Score for Neural Network Classifiers](https://mlsanity.com/paper/1709.09844/) (2017)
- [Deep Probability Estimation](https://mlsanity.com/paper/2111.10734/) (2021)
- [Capture Uncertainties in Deep Neural Networks for Safe Operation of Autonomous Driving Vehicles](https://mlsanity.com/paper/2108.05118/) (2021)

## Built on by (most cited first, 8 of 8)

- [Calibrating Uncertainty for Semi-Supervised Crowd Counting](https://mlsanity.com/paper/2308.09887/) (2023)
- [MetaSSL: A General Heterogeneous Loss for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2509.01144/) (2025)
- [Spatial Diffusion for Cell Layout Generation](https://mlsanity.com/paper/2409.03106/) (2024)
- [MATCH: Multi-faceted Adaptive Topo-Consistency for Semi-Supervised Histopathology Segmentation](https://mlsanity.com/paper/2510.01532/) (2025)
- [ORCE: Order-Aware Alignment of Verbalized Confidence in Large Language Models](https://mlsanity.com/paper/2605.12446/) (2026)
- [Uncertainty Estimation for Pretrained Medical Image Registration Models via Transformation Equivariance](https://mlsanity.com/paper/2509.23355/) (2025)
- [Learning Topological Representations for Deep Image Understanding](https://mlsanity.com/paper/2403.15361/) (2024)
- [Multi-Channel Uncertainty-Weighted Score Matching for Conditional Diffusion in Medical UDA](https://mlsanity.com/paper/2509.22476/) (2025)

## Abstract (arXiv)

Overconfidence is a common issue for deep neural networks, limiting their deployment in real-world applications. To better estimate confidence, existing methods mostly focus on fully-supervised scenarios and rely on training labels. In this paper, we propose the first confidence estimation method for a semi-supervised setting, when most training labels are unavailable. We stipulate that even with limited training labels, we can still reasonably approximate the confidence of model on unlabeled samples by inspecting the prediction consistency through the training process. We use training consistency as a surrogate function and propose a consistency ranking loss for confidence estimation. On both image classification and segmentation tasks, our method achieves state-of-the-art performances in confidence estimation. Furthermore, we show the benefit of the proposed method through a downstream active learning task. The code is available at https://github.com/TopoXLab/consistency-ranking-loss