Evaluating Uncertainty Calibration for Open-Set Recognition
Not on the map. Why?
Abstract (arXiv)
Despite achieving enormous success in predictive accuracy for visual classification problems, deep neural networks (DNNs) suffer from providing overconfident probabilities on out-of-distribution (OOD) data. Yet, accurate uncertainty estimation is crucial for safe and reliable robot autonomy. In this paper, we evaluate popular calibration techniques for open-set conditions in a way that is distinctly different from the conventional evaluation of calibration methods on OOD data. Our results show that closed-set DNN calibration approaches are much less effective for open-set recognition, which highlights the need to develop new DNN calibration methods to address this problem.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 23 of its 32 references that are in the map. First posted on arXiv 15 May 2022.
Builds on
23 of 32 references.
References · 23
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2016Densely Connected Convolutional Networkscited by 6,720
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
- 2015Towards Open Set Deep Networkscited by 538
- 2021Revisiting the Calibration of Modern Neural Networkscited by 390
- 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
- 2015Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understandingcited by 218
11 more
- 2018Classification-Reconstruction Learning for Open-Set Recognitioncited by 146
- 2019C2AE: Class Conditioned Auto-Encoder for Open-set Recognitioncited by 142
- 2021Learning Placeholders for Open-Set Recognitioncited by 121
- 2020Hybrid Models for Open Set Recognitioncited by 109
- 2018The Limits and Potentials of Deep Learning for Roboticscited by 104
- 2020Multivariate Confidence Calibration for Object Detectioncited by 98
- 2021Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibrationcited by 79
- 2017Failing to Learn: Autonomously Identifying Perception Failures for Self-driving Carscited by 28
- 2015Place Categorization and Semantic Mapping on a Mobile Robotcited by 12
- 2021An Uncertainty Estimation Framework for Probabilistic Object Detectioncited by 4
- 2021Open-World Active Learning with Stacking Ensemble for Self-Driving Carscited by 1
Built on by
Citing papers per year: 2022: 1.
1 paper.