mlsanityData to 7 Oct 2026

Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis

arXiv 2111.04138 ↗First posted 7 Nov 2021cs.CV cs.AI cs.CLcited by 65

Not on the map. Why?

Abstract (arXiv)

We describe a novel attribution method which is grounded in Sensitivity Analysis and uses Sobol indices. Beyond modeling the individual contributions of image regions, Sobol indices provide an efficient way to capture higher-order interactions between image regions and their contributions to a neural network's prediction through the lens of variance. We describe an approach that makes the computation of these indices efficient for high-dimensional problems by using perturbation masks coupled with efficient estimators to handle the high dimensionality of images. Importantly, we show that the proposed method leads to favorable scores on standard benchmarks for vision (and language models) while drastically reducing the computing time compared to other black-box methods -- even surpassing the accuracy of state-of-the-art white-box methods which require access to internal representations. Our code is freely available: https://github.com/fel-thomas/Sobol-Attribution-Method

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 33 of its 70 references that are in the map. First posted on arXiv 7 Nov 2021.

Builds on

33 of 70 references.

References · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  7. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  9. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  10. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  11. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  12. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
21 more
  1. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  2. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  3. 2018Sanity Checks for Saliency Mapscited by 1,149
  4. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  5. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  6. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  7. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  8. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
  9. 2017Interpretation of Neural Networks is Fragilecited by 443
  10. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 440
  11. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  12. 2016Top-down Neural Attention by Excitation Backpropcited by 318
  13. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 295
  14. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 257
  15. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 185
  16. 2019Sanity Checks for Saliency Metricscited by 119
  17. 2019XRAI: Better Attributions Through Regionscited by 116
  18. 2017Explaining Recurrent Neural Network Predictions in Sentiment Analysiscited by 87
  19. 2021Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answeringcited by 71
  20. 2016"What is Relevant in a Text Document?": An Interpretable Machine Learning Approachcited by 53
  21. 2019Computing Linear Restrictions of Neural Networkscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 11; 2023: 14; 2024: 9; 2025: 18; 2026: 12.

65 papers.

Sort:

By area:

  1. 2022CRAFT: Concept Recursive Activation FacTorization for Explainabilitycited by 177
  2. 2023A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimationcited by 108
  3. 2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
  4. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 67
  5. 2023On the Foundations of Shortcut Learningcited by 64
  6. 2025Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignmentcited by 62
  7. 2022Harmonizing the object recognition strategies of deep neural networks with humanscited by 59
  8. 2022Don't Lie to Me! Robust and Efficient Explainability with Verified Perturbation Analysiscited by 38
  9. 2022Making Sense of Dependence: Efficient Black-box Explanations Using Dependence Measurecited by 33
  10. 2022Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learningcited by 32
  11. 2023Unlocking Feature Visualization for Deeper Networks with MAgnitude Constrained Optimizationcited by 26
  12. 2022Human Interpretation of Saliency-based Explanation Over Textcited by 25
38 more
  1. 2022Xplique: A Deep Learning Explainability Toolboxcited by 24
  2. 2023COCKATIEL: COntinuous Concept ranKed ATtribution with Interpretable ELements for explaining neural net classifiers on NLP taskscited by 16
  3. 2022Deletion and Insertion Tests in Regression Modelscited by 14
  4. 2023Natural Example-Based Explainability: a Surveycited by 13
  5. 2023Diffusion Models as Artists: Are we Closing the Gap between Humans and Machines?cited by 10
  6. 2025Better artificial intelligence does not mean better models of biologycited by 10
  7. 2026Cross-Modal Redundancy and the Geometry of Vision-Language Embeddingscited by 9
  8. 2023Local Concept Embeddings for Analysis of Concept Distributions in Vision DNN Feature Spacescited by 7
  9. 2023An adversarial attack approach for eXplainable AI evaluation on deepfake detection modelscited by 7
  10. 2022On the explainable properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspectivecited by 6
  11. 2024Towards Quantitative Evaluation of Explainable AI Methods for Deepfake Detectioncited by 6
  12. 2023Assessment of the Reliablity of a Model's Decision by Generalizing Attribution to the Wavelet Domaincited by 5
  13. 2024Feature Accentuation: Revealing 'What' Features Respond to in Natural Imagescited by 5
  14. 2022Quantile-constrained Wasserstein projections for robust interpretability of numerical and machine learning modelscited by 4
  15. 2023Neighboring Words Affect Human Interpretation of Saliency Explanationscited by 4
  16. 2024Space-scale Exploration of the Poor Reliability of Deep Learning Models: the Case of the Remote Sensing of Rooftop Photovoltaic Systemscited by 4
  17. 2025Unveil Sources of Uncertainty: Feature Contribution to Conformal Prediction Intervalscited by 4
  18. 2025Evaluating Neuron Explanations: A Unified Framework with Sanity Checkscited by 4
  19. 2026Bi-Orthogonal Factor Decomposition for Vision Transformerscited by 4
  20. 2022A survey of Identification and mitigation of Machine Learning algorithmic biases in Image Analysiscited by 3
  21. 2023Saliency strikes back: How filtering out high frequencies improves white-box explanationscited by 3
  22. 2023Trainable Noise Model as an XAI evaluation method: application on Sobol for remote sensing image segmentationcited by 3
  23. 2024One Wave To Explain Them All: A Unifying Perspective On Feature Attributioncited by 3
  24. 2025A Geometric Unification of Concept Learning with Concept Conescited by 3
  25. 2026Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifoldscited by 3
  26. 2023Can We Reliably Improve the Robustness to Image Acquisition of Remote Sensing of PV Systems?cited by 2
  27. 2024The Overfocusing Bias of Convolutional Neural Networks: A Saliency-Guided Regularization Approachcited by 2
  28. 2024GIFT: A Framework Towards Global Interpretable Faithful Textual Explanations of Vision Classifierscited by 2
  29. 2025Improving the Perturbation-Based Explanation of Deepfake Detectors Through the Use of Adversarially-Generated Samplescited by 2
  30. 2025Exploring specialization and sensitivity of convolutional neural networks in the context of simultaneous image augmentationscited by 2
  31. 2025Improving Perturbation-based Explanations by Understanding the Role of Uncertainty Calibrationcited by 2
  32. 2026Spectral Integrated Gradients for Coarse-to-Fine Feature Attributioncited by 2
  33. 2024A Mechanistic Study of Transformers Training Dynamicscited by 1
  34. 2025Measuring the (Un)Faithfulness of Concept-Based Explanationscited by 1
  35. 2025FaCT: Faithful Concept Traces for Explaining Neural Network Decisionscited by 1
  36. 2025Interpreto: An Explainability Library for Transformerscited by 1
  37. 2025Back to the Baseline: Examining Baseline Effects on Explainability Metricscited by 1
  38. 2026Hoeffding Concept Bottleneck Models with Applications to Overhead Imagescited by 1

50 most cited of 65 All 65 (JSON)