# Generative Perturbation Analysis for Probabilistic Black-Box Anomaly Attribution

arXiv: https://arxiv.org/abs/2308.04708 · HTML: https://mlsanity.com/paper/2308.04708/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 50 references that are in the map.
- First posted on arXiv 9 Aug 2023.

## Builds on (most cited first, 24 of 24)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [The many Shapley values for model explanation](https://mlsanity.com/paper/1908.08474/) (2019)
- [Understanding Deep Networks via Extremal Perturbations and Smooth Masks](https://mlsanity.com/paper/1910.08485/) (2019)
- [Problems with Shapley-value-based explanations as feature importance measures](https://mlsanity.com/paper/2002.11097/) (2020)
- [Reliable Post hoc Explanations: Modeling Uncertainty in Explainability](https://mlsanity.com/paper/2008.05030/) (2020)
- [What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research](https://mlsanity.com/paper/2102.07817/) (2021)
- [Do Feature Attribution Methods Correctly Attribute Features?](https://mlsanity.com/paper/2104.14403/) (2021)
- [Explaining the Explainer: A First Theoretical Analysis of LIME](https://mlsanity.com/paper/2001.03447/) (2020)
- [Explaining Black-Box Algorithms Using Probabilistic Contrastive Counterfactuals](https://mlsanity.com/paper/2103.11972/) (2021)
- [BayLIME: Bayesian Local Interpretable Model-Agnostic Explanations](https://mlsanity.com/paper/2012.03058/) (2020)
- [Interpretable, Multidimensional, Multimodal Anomaly Detection with Negative Sampling for Detection of Device Failure](https://mlsanity.com/paper/2007.10088/) (2020)
- [Explaining Deep Learning Models - A Bayesian Non-parametric Approach](https://mlsanity.com/paper/1811.03422/) (2018)
- [A Unified Taylor Framework for Revisiting Attribution Methods](https://mlsanity.com/paper/2008.09695/) (2020)
- [An Interpretable Probabilistic Approach for Demystifying Black-box Predictive Models](https://mlsanity.com/paper/2007.10668/) (2020)
- [A general-purpose method for applying Explainable AI for Anomaly Detection](https://mlsanity.com/paper/2207.11564/) (2022)
- [Anomaly Attribution with Likelihood Compensation](https://mlsanity.com/paper/2208.10679/) (2022)
- [ACE -- An Anomaly Contribution Explainer for Cyber-Security Applications](https://mlsanity.com/paper/1912.00314/) (2019)
- [Black-Box Anomaly Attribution](https://mlsanity.com/paper/2305.18440/) (2023)

## Built on by (most cited first, 6 of 6)

- [Exploring Energy Landscapes for Minimal Counterfactual Explanations: Applications in Cybersecurity and Beyond](https://mlsanity.com/paper/2503.18185/) (2025)
- [Causal-discovery-based root-cause analysis and its application in time-series prediction error diagnosis](https://mlsanity.com/paper/2411.06990/) (2024)
- [SMA: Who Said That? Auditing Membership Leakage in Semi-Black-box RAG Controlling](https://mlsanity.com/paper/2508.09105/) (2025)
- [Concerning Uncertainty -- A Systematic Survey of Uncertainty-Aware XAI](https://mlsanity.com/paper/2603.26838/) (2026)
- [Wafer Defect Root Cause Analysis with Partial Trajectory Regression](https://mlsanity.com/paper/2507.20357/) (2025)
- [Sequence-Aware Inline Measurement Attribution for Good-Bad Wafer Diagnosis](https://mlsanity.com/paper/2507.20364/) (2025)

## Abstract (arXiv)

We address the task of probabilistic anomaly attribution in the black-box regression setting, where the goal is to compute the probability distribution of the attribution score of each input variable, given an observed anomaly. The training dataset is assumed to be unavailable. This task differs from the standard XAI (explainable AI) scenario, since we wish to explain the anomalous deviation from a black-box prediction rather than the black-box model itself. We begin by showing that mainstream model-agnostic explanation methods, such as the Shapley values, are not suitable for this task because of their ``deviation-agnostic property.'' We then propose a novel framework for probabilistic anomaly attribution that allows us to not only compute attribution scores as the predictive mean but also quantify the uncertainty of those scores. This is done by considering a generative process for perturbations that counter-factually bring the observed anomalous observation back to normalcy. We introduce a variational Bayes algorithm for deriving the distributions of per variable attribution scores. To the best of our knowledge, this is the first probabilistic anomaly attribution framework that is free from being deviation-agnostic.