mlsanityData to 7 Oct 2026

Explaining Predictive Uncertainty by Exposing Second-Order Effects

arXiv 2401.17441 ↗First posted 30 Jan 2024cs.LG cs.AI stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

Explainable AI has brought transparency into complex ML blackboxes, enabling, in particular, to identify which features these models use for their predictions. So far, the question of explaining predictive uncertainty, i.e. why a model 'doubts', has been scarcely studied. Our investigation reveals that predictive uncertainty is dominated by second-order effects, involving single features or product interactions between them. We contribute a new method for explaining predictive uncertainty based on these second-order effects. Computationally, our method reduces to a simple covariance computation over a collection of first-order explanations. Our method is generally applicable, allowing for turning common attribution techniques (LRP, Gradient x Input, etc.) into powerful second-order uncertainty explainers, which we call CovLRP, CovGI, etc. The accuracy of the explanations our method produces is demonstrated through systematic quantitative evaluations, and the overall usefulness of our method is demonstrated via two practical showcases.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 33 of its 60 references that are in the map. First posted on arXiv 30 Jan 2024.

Builds on

33 of 60 references.

References · 33

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  7. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  8. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  9. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  10. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  11. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  12. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
21 more
  1. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  2. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  3. 2019Deep Evidential Regressioncited by 354
  4. 2019Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentationcited by 162
  5. 2019The Shapley Taylor Interaction Indexcited by 143
  6. 2020Higher-Order Explanations of Graph Neural Networks via Relevant Walkscited by 138
  7. 2020Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmarkcited by 105
  8. 2020Explaining Explanations: Axiomatic Feature Interactions for Deep Networkscited by 103
  9. 2020Getting a CLUE: A Method for Explaining Uncertainty Estimatescited by 92
  10. 2017Uncertainty-Aware Reinforcement Learning for Collision Avoidancecited by 86
  11. 2018Bayesian Uncertainty Estimation for Batch Normalized Deep Networkscited by 70
  12. 2019Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Controlcited by 60
  13. 2023Explaining Predictive Uncertainty with Information Theoretic Shapley Valuescited by 36
  14. 2019From Clustering to Cluster Explanations via Neural Networkscited by 34
  15. 2020Building and Interpreting Deep Similarity Modelscited by 34
  16. 2021Toward Explainable AI for Regression Modelscited by 30
  17. 2023Explainable AI for Time Series via Virtual Inspection Layerscited by 23
  18. 2021Monte Carlo DropBlock for Modelling Uncertainty in Object Detectioncited by 20
  19. 2019On the Explanation of Machine Learning Predictions in Clinical Gait Analysiscited by 15
  20. 2023Model-agnostic variable importance for predictive uncertainty: an entropy-based approachcited by 6
  21. 2017Sensitivity Analysis for Predictive Uncertainty in Bayesian Neural Networkscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 6.

9 papers.

  1. 2025Unveil Sources of Uncertainty: Feature Contribution to Conformal Prediction Intervalscited by 4
  2. 2026Concerning Uncertainty -- A Systematic Survey of Uncertainty-Aware XAIcited by 2
  3. 2026Normalized Relevance Measure as a Unifying Framework to Explain Neural Network Latent Structurescited by 2
  4. 2026CIRCUS: Circuit Consensus under Uncertainty via Stability Ensemblescited by 1
  5. 2025Conceptualizing Uncertainty: A Concept-based Approach to Explaining Uncertaintycited by 0
  6. 2025Fast and Accurate Explanations of Distance-Based Classifiers by Uncovering Latent Explanatory Structurescited by 0
  7. 2026No Single Metric Tells the Whole Story: A Multi-Dimensional Evaluation Framework for Uncertainty Attributionscited by 0
  8. 2026Uncertainty Gating for Cost-Aware Explainable Artificial Intelligencecited by 0
  9. 2026Uncertainty-Aware End-to-End AI Weather Forecasting: Disentangling Observation and Model Contributionscited by 0