mlsanityData to 7 Oct 2026

Don't Lie to Me! Robust and Efficient Explainability with Verified Perturbation Analysis

arXiv 2202.07728 ↗First posted 15 Feb 2022cs.CVcited by 38

Not on the map. Why?

Abstract (arXiv)

A variety of methods have been proposed to try to explain how deep neural networks make their decisions. Key to those approaches is the need to sample the pixel space efficiently in order to derive importance maps. However, it has been shown that the sampling methods used to date introduce biases and other artifacts, leading to inaccurate estimates of the importance of individual pixels and severely limit the reliability of current explainability methods. Unfortunately, the alternative -- to exhaustively sample the image space is computationally prohibitive. In this paper, we introduce EVA (Explaining using Verified perturbation Analysis) -- the first explainability method guarantee to have an exhaustive exploration of a perturbation space. Specifically, we leverage the beneficial properties of verified perturbation analysis -- time efficiency, tractability and guaranteed complete coverage of a manifold -- to efficiently characterize the input variables that are most likely to drive the model decision. We evaluate the approach systematically and demonstrate state-of-the-art results on multiple benchmarks.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 49 of its 78 references that are in the map. First posted on arXiv 15 Feb 2022.

Builds on

49 of 78 references.

References · 49

  1. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  7. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  8. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  9. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  10. 2018Sanity Checks for Saliency Mapscited by 1,149
  11. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  12. 2014Striving for Simplicity: The All Convolutional Netcited by 983
37 more
  1. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 936
  2. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  3. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
  4. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  5. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  6. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
  7. 2017Interpretation of Neural Networks is Fragilecited by 443
  8. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 439
  9. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  10. 2017The (Un)reliability of saliency methodscited by 331
  11. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  12. 2017Formal Verification of Piece-Wise Linear Feed-Forward Neural Networkscited by 241
  13. 2021Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Complete and Incomplete Neural Network Robustness Verificationcited by 224
  14. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 185
  15. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  16. 2018Abduction-Based Explanations for Machine Learning Modelscited by 159
  17. 2020Reliable Post hoc Explanations: Modeling Uncertainty in Explainabilitycited by 125
  18. 2019A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networkscited by 124
  19. 2021Counterfactual Explanations Can Be Manipulatedcited by 113
  20. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  21. 2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
  22. 2021The effectiveness of feature attribution methods and its correlation with automatic evaluation scorescited by 87
  23. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 86
  24. 2021Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysiscited by 65
  25. 2020Evaluations and Methods for Explanation through Robustness Analysiscited by 51
  26. 2020Generative causal explanations of black-box classifierscited by 41
  27. 2020Proper Network Interpretability Helps Adversarial Robustness in Classificationcited by 41
  28. 2019Do Explanations Reflect Decisions? A Machine-centric Strategy to Quantify the Performance of Explainability Algorithmscited by 32
  29. 2021On the Sensitivity and Stability of Model Interpretations in NLPcited by 30
  30. 2021On Baselines for Local Feature Attributionscited by 29
  31. 2021On Locality of Local Explanation Modelscited by 25
  32. 2020Learning Models for Actionable Recoursecited by 18
  33. 2017Evaluating Robustness of Neural Networks with Mixed Integer Programmingcited by 14
  34. 2019Computing Linear Restrictions of Neural Networkscited by 14
  35. 2018Noise-adding Methods of Saliency Map as Series of Higher Order Partial Derivativecited by 11
  36. 2021Developments and applications of Shapley effects to reliability-oriented sensitivity analysis with correlated inputscited by 4
  37. 2021Coalitional strategies for efficient individual prediction explanationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 8; 2024: 11; 2025: 7; 2026: 7.

38 papers.

Sort:

By area:

  1. 2022CRAFT: Concept Recursive Activation FacTorization for Explainabilitycited by 177
  2. 2023A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimationcited by 108
  3. 2022Harmonizing the object recognition strategies of deep neural networks with humanscited by 59
  4. 2022Towards Formal XAI: Formally Approximate Minimal Explanations of Neural Networkscited by 42
  5. 2024Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarkscited by 28
  6. 2023Unlocking Feature Visualization for Deeper Networks with MAgnitude Constrained Optimizationcited by 26
  7. 2022Xplique: A Deep Learning Explainability Toolboxcited by 24
  8. 2023Formally Explaining Neural Networks within Reactive Systemscited by 24
  9. 2024Local vs. Global Interpretability: A Computational Complexity Perspectivecited by 23
  10. 2023Natural Example-Based Explainability: a Surveycited by 13
  11. 2025Explain Yourself, Briefly! Self-Explaining Neural Networks with Concise Sufficient Reasonscited by 11
  12. 2023Diffusion Models as Artists: Are we Closing the Gap between Humans and Machines?cited by 10
26 more
  1. 2023Keep the Faith: Faithful Explanations in Convolutional Neural Networks for Case-Based Reasoningcited by 10
  2. 2026Formal Mechanistic Interpretability: Automated Circuit Discovery with Provable Guaranteescited by 10
  3. 2025Tokenize Image Patches: Global Context Fusion for Effective Haze Removal in Large Imagescited by 7
  4. 2022On the explainable properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspectivecited by 6
  5. 2024SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generationcited by 6
  6. 2023Assessment of the Reliablity of a Model's Decision by Generalizing Attribution to the Wavelet Domaincited by 5
  7. 2024Feature Accentuation: Revealing 'What' Features Respond to in Natural Imagescited by 5
  8. 2024A Study of Nationality Bias in Names and Perplexity using Off-the-Shelf Affect-related Tweet Classifierscited by 4
  9. 2025Efficient Robust Conformal Prediction via Lipschitz-Bounded Networkscited by 4
  10. 2024Mitigating Low-Frequency Bias: Feature Recalibration and Frequency Attention Regularization for Adversarial Robustnesscited by 3
  11. 2026FAME: Formal Abstract Minimal Explanation for Neural Networkscited by 3
  12. 2024Explainable Image Recognition via Enhanced Slot-attention Based Classifiercited by 2
  13. 2024OMENN: One Matrix to Explain Neural Networkscited by 2
  14. 2025Interpretable 3D Neural Object Volumes for Robust Conceptual Reasoningcited by 2
  15. 2025A Survey on Interpretability in Visual Recognitioncited by 2
  16. 2023Deep Natural Language Feature Learning for Interpretable Predictioncited by 1
  17. 2026Provably Explaining Neural Additive Modelscited by 1
  18. 2024Tensor Polynomial Additive Modelcited by 0
  19. 2024Understanding Inhibition Through Maximally Tense Imagescited by 0
  20. 2024Exploring Channel Distinguishability in Local Neighborhoods of the Model Space in Quantum Neural Networkscited by 0
  21. 2025Towards Robust and Generalizable Gerchberg Saxton based Physics Inspired Neural Networks for Computer Generated Holography: A Sensitivity Analysis Frameworkcited by 0
  22. 2025Learning Conformal Explainers for Image Classifierscited by 0
  23. 2026Hidden Monotonicity: Explaining Deep Neural Networks via their DC Decompositioncited by 0
  24. 2026Learn to Rank: Visual Attribution by Learning Importance Rankingcited by 0
  25. 2026From Weight Perturbation to Feature Attribution for Explaining Fully Connected Neural Networkscited by 0
  26. 2026A unified framework for global and local interpretability using adaptive derivative-ordered random explanationcited by 0