# Unveil Sources of Uncertainty: Feature Contribution to Conformal Prediction Intervals

arXiv: https://arxiv.org/abs/2505.13118 · HTML: https://mlsanity.com/paper/2505.13118/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 76 references that are in the map.
- First posted on arXiv 19 May 2025.

## Builds on (most cited first, 29 of 29)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [The many Shapley values for model explanation](https://mlsanity.com/paper/1908.08474/) (2019)
- [Learning to Explain: An Information-Theoretic Perspective on Model Interpretation](https://mlsanity.com/paper/1802.07814/) (2018)
- [Explaining by Removing: A Unified Framework for Model Explanation](https://mlsanity.com/paper/2011.14878/) (2020)
- [FastSHAP: Real-Time Shapley Value Estimation](https://mlsanity.com/paper/2107.07436/) (2021)
- [Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models](https://mlsanity.com/paper/2011.01625/) (2020)
- [Getting a CLUE: A Method for Explaining Uncertainty Estimates](https://mlsanity.com/paper/2006.06848/) (2020)
- [Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis](https://mlsanity.com/paper/2111.04138/) (2021)
- [shapiq: Shapley Interactions for Machine Learning](https://mlsanity.com/paper/2410.01649/) (2024)
- [Explaining Predictive Uncertainty with Information Theoretic Shapley Values](https://mlsanity.com/paper/2306.05724/) (2023)
- [RKHS-SHAP: Shapley Values for Kernel Methods](https://mlsanity.com/paper/2110.09167/) (2021)
- [Boosted Conformal Prediction Intervals](https://mlsanity.com/paper/2406.07449/) (2024)
- [Diverse, Global and Amortised Counterfactual Explanations for Uncertainty Estimates](https://mlsanity.com/paper/2112.02646/) (2021)
- [Calibrated Explanations: with Uncertainty Information and Counterfactuals](https://mlsanity.com/paper/2305.02305/) (2023)
- [Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metrics](https://mlsanity.com/paper/2409.16756/) (2024)
- [Don't be fooled: label leakage in explanation methods and the importance of their quantitative evaluation](https://mlsanity.com/paper/2302.12893/) (2023)
- [Quantifying and Explaining Machine Learning Uncertainty in Predictive Process Monitoring: An Operations Research Perspective](https://mlsanity.com/paper/2304.06412/) (2023)
- [Explaining Predictive Uncertainty by Exposing Second-Order Effects](https://mlsanity.com/paper/2401.17441/) (2024)
- [Uncertainty quantification for probabilistic machine learning in earth observation using conformal prediction](https://mlsanity.com/paper/2401.06421/) (2024)
- [Attribution of Predictive Uncertainties in Classification Models](https://mlsanity.com/paper/2107.08756/) (2021)
- [Proportional marginal effects for global sensitivity analysis](https://mlsanity.com/paper/2210.13065/) (2022)
- [Why You Should Not Trust Interpretations in Machine Learning: Adversarial Attacks on Partial Dependence Plots](https://mlsanity.com/paper/2404.18702/) (2024)
- [Quantile-constrained Wasserstein projections for robust interpretability of numerical and machine learning models](https://mlsanity.com/paper/2209.11539/) (2022)
- [Approximating Score-based Explanation Techniques Using Conformal Regression](https://mlsanity.com/paper/2308.11975/) (2023)
- [Identifying Drivers of Predictive Aleatoric Uncertainty](https://mlsanity.com/paper/2312.07252/) (2023)

## Built on by (most cited first, 5 of 5)

- [Interpretable and Explainable Surrogate Modeling for Simulations: A State-of-the-Art Survey and Perspectives on Explainable AI for Decision-Making](https://mlsanity.com/paper/2604.14240/) (2026)
- [Learning Conformal Explainers for Image Classifiers](https://mlsanity.com/paper/2509.21209/) (2025)
- [Informative Perturbation Selection for Uncertainty-Aware Post-hoc Explanations](https://mlsanity.com/paper/2603.14894/) (2026)
- [Evaluating Black-Box Vulnerabilities with Wasserstein-Constrained Data Perturbations](https://mlsanity.com/paper/2603.15867/) (2026)
- [A Hierarchy of Entropy-Shapley Games for Multivariate Predictive Uncertainty](https://mlsanity.com/paper/2609.35217/) (2026)

## Abstract (arXiv)

Cooperative game theory methods, notably Shapley values, have significantly enhanced machine learning (ML) interpretability. However, existing explainable AI (XAI) frameworks mainly attribute average model predictions, overlooking predictive uncertainty. This work addresses that gap by proposing a novel, model-agnostic uncertainty attribution (UA) method grounded in conformal prediction (CP). By defining cooperative games where CP interval properties-such as width and bounds-serve as value functions, we systematically attribute predictive uncertainty to input features. Extending beyond the traditional Shapley values, we use the richer class of Harsanyi allocations, and in particular the proportional Shapley values, which distribute attribution proportionally to feature importance. We propose a Monte Carlo approximation method with robust statistical guarantees to address computational feasibility, significantly improving runtime efficiency. Our comprehensive experiments on synthetic benchmarks and real-world datasets demonstrate the practical utility and interpretative depth of our approach. By combining cooperative game theory and conformal prediction, we offer a rigorous, flexible toolkit for understanding and communicating predictive uncertainty in high-stakes ML applications.