mlsanityData to 7 Oct 2026

Uncertainty Surrogates for Deep Learning

arXiv 2104.08147 ↗First posted 16 Apr 2021cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper we introduce a novel way of estimating prediction uncertainty in deep networks through the use of uncertainty surrogates. These surrogates are features of the penultimate layer of a deep network that are forced to match predefined patterns. The patterns themselves can be, among other possibilities, a known visual symbol. We show how our approach can be used for estimating uncertainty in prediction and out-of-distribution detection. Additionally, the surrogates allow for interpretability of the ability of the deep network to learn and at the same time lend robustness against adversarial attacks. Despite its simplicity, our approach is superior to the state-of-the-art approaches on standard metrics as well as computational efficiency and ease of implementation. A wide range of experiments are performed on standard datasets to prove the efficacy of our approach.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 16 Apr 2021.

Builds on

26 of 37 references.

References · 26

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2013Intriguing properties of neural networkscited by 5,114
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  4. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  5. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  6. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  7. 2018Exploration by Random Network Distillationcited by 966
  8. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  9. 2017Selective Classification for Deep Neural Networkscited by 799
  10. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 728
  11. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  12. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
14 more
  1. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  2. 2020Training independent subnetworks for robust predictioncited by 177
  3. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  4. 2018High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approachcited by 117
  5. 2020Masksembles for Uncertainty Estimationcited by 88
  6. 2018Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?cited by 62
  7. 2017A causal framework for explaining the predictions of black-box sequence-to-sequence modelscited by 58
  8. 2017Deep and Confident Prediction for Time Series at Ubercited by 50
  9. 2020Hydra: Preserving Ensemble Diversity for Model Distillationcited by 31
  10. 2018Confidence Scoring Using Whitebox Meta-models with Linear Classifier Probescited by 21
  11. 2019Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernelcited by 19
  12. 2018A Less Biased Evaluation of Out-of-distribution Sample Detectorscited by 14
  13. 2019Modelling heterogeneous distributions with an Uncountable Mixture of Asymmetric Laplacianscited by 9
  14. 2018Prior Networks for Detection of Adversarial Attackscited by 2

Built on by

Not cited yet.