Frustratingly Easy Uncertainty Estimation for Distribution Shift
Not on the map. Why?
Abstract (arXiv)
Distribution shift is an important concern in deep image classification, produced either by corruption of the source images, or a complete change, with the solution involving domain adaptation. While the primary goal is to improve accuracy under distribution shift, an important secondary goal is uncertainty estimation: evaluating the probability that the prediction of a model is correct. While improving accuracy is hard, uncertainty estimation turns out to be frustratingly easy. Prior works have appended uncertainty estimation into the model and training paradigm in various ways. Instead, we show that we can estimate uncertainty by simply exposing the original model to corrupted images, and performing simple statistical calibration on the image outputs. Our frustratingly easy methods demonstrate superior performance on a wide range of distribution shifts as well as on unsupervised domain adaptation tasks, measured through extensive experimentation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 34 of its 51 references that are in the map. First posted on arXiv 7 Jun 2021.
Builds on
34 of 51 references.
References · 34
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2015Domain-Adversarial Training of Neural Networkscited by 3,044
- 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
- 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
- 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,329
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
- 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,188
- 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,013
22 more
- 2018Deep Anomaly Detection with Outlier Exposurecited by 942
- 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 901
- 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 776
- 2016Deep Transfer Learning with Joint Adaptation Networkscited by 612
- 2019Measuring Calibration in Deep Learningcited by 446
- 2017VisDA: The Visual Domain Adaptation Challengecited by 435
- 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
- 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 364
- 2019Bridging Theory and Algorithm for Domain Adaptationcited by 332
- 2018Generalisation in humans and deep neural networkscited by 237
- 2020Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shiftcited by 236
- 2017Multiplicative Normalizing Flows for Variational Bayesian Neural Networkscited by 157
- 2018Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batchescited by 133
- 2020Improving model calibration with accuracy versus uncertainty optimizationcited by 123
- 2016Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriorscited by 82
- 2020Distribution-free binary classification: prediction sets, confidence intervals and calibrationcited by 79
- 2020Intra Order-preserving Functions for Calibration of Multi-Class Neural Networkscited by 65
- 2016Examining the Impact of Blur on Recognition by Convolutional Networkscited by 58
- 2020Calibrated Prediction with Covariate Shift via Unsupervised Domain Adaptationcited by 43
- 2020Transferable Calibration with Lower Bias and Variance in Domain Adaptationcited by 41
- 2020An Empirical Evaluation on Robustness and Uncertainty of Regularization Methodscited by 28
- 2020Calibrating Deep Neural Network Classifiers on Out-of-Distribution Datasetscited by 7
Built on by
Citing papers per year: 2023: 1.
1 paper.