Tailored Uncertainty Estimation for Deep Learning Systems
Not on the map. Why?
Abstract (arXiv)
Uncertainty estimation bears the potential to make deep learning (DL) systems more reliable. Standard techniques for uncertainty estimation, however, come along with specific combinations of strengths and weaknesses, e.g., with respect to estimation quality, generalization abilities and computational complexity. To actually harness the potential of uncertainty quantification, estimators are required whose properties closely match the requirements of a given use case. In this work, we propose a framework that, firstly, structures and shapes these requirements, secondly, guides the selection of a suitable uncertainty estimation method and, thirdly, provides strategies to validate this choice and to uncover structural weaknesses. By contributing tailored uncertainty estimation in this sense, our framework helps to foster trustworthy DL systems. Moreover, it anticipates prospective machine learning regulations that require, e.g., in the EU, evidences for the technical appropriateness of machine learning systems. Our framework provides such evidences for system components modeling uncertainty.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 52 of its 121 references that are in the map. First posted on arXiv 29 Apr 2022.
Builds on
52 of 121 references.
References · 52
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
- 2016Concrete Problems in AI Safetycited by 1,642
- 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
- 2018Model Cards for Model Reportingcited by 1,080
38 more
- 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
- 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
- 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
- 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
- 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
- 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
- 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 422
- 2019Deep Evidential Regressioncited by 355
- 2017DeepXplore: Automated Whitebox Testing of Deep Learning Systemscited by 311
- 2019Verified Uncertainty Calibrationcited by 309
- 2018A Probabilistic U-Net for Segmentation of Ambiguous Imagescited by 283
- 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
- 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
- 2017Inverse Reward Designcited by 249
- 2019Machine Learning Testing: Survey, Landscapes and Horizonscited by 218
- 2018Bounding Box Regression with Uncertainty for Accurate Object Detectioncited by 143
- 2018Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flowcited by 135
- 2019Meta-Sim: Learning to Generate Synthetic Datasetscited by 128
- 2019A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networkscited by 125
- 2018The Limits and Potentials of Deep Learning for Roboticscited by 104
- 2020Uncertainty-based Traffic Accident Anticipation with Spatio-Temporal Relational Learningcited by 91
- 2020Masksembles for Uncertainty Estimationcited by 88
- 2019Sampling-free Epistemic Uncertainty Estimation Using Approximated Variance Propagationcited by 74
- 2019Unsupervised Traffic Accident Detection in First-Person Videoscited by 70
- 2018TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzingcited by 68
- 2021Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantificationcited by 57
- 2016SqueezeDet: Unified, Small, Low Power Fully Convolutional Neural Networks for Real-Time Object Detection for Autonomous Drivingcited by 55
- 2018Probabilistic Object Detection: Definition and Evaluationcited by 55
- 2019Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Trafficcited by 50
- 2018On Testing Machine Learning Programscited by 48
- 2019Accurate Uncertainty Estimation and Decomposition in Ensemble Learningcited by 42
- 2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safetycited by 35
- 2018FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformitycited by 31
- 2020Safety Concerns and Mitigation Approaches Regarding the Use of Deep Learning in Safety-Critical Perception Taskscited by 31
- 2018Efficient Decentralized Deep Learning by Dynamic Model Averagingcited by 30
- 2021Uncertainty Quantification 360: A Holistic Toolkit for Quantifying and Communicating the Uncertainty of AIcited by 22
- 2019Application of Decision Rules for Handling Class Imbalance in Semantic Segmentationcited by 9
- 2021A Novel Regression Loss for Non-Parametric Uncertainty Optimizationcited by 4
50 most cited of 52 All 52 (JSON)
Built on by
Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 1; 2026: 1.
3 papers.