mlsanityData to 14 Oct 2026

Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond

arXiv 2202.06861 ↗First posted 14 Feb 2022cs.LGcited by 171

Not on the map. Why?

Abstract (arXiv)

The evaluation of explanation methods is a research topic that has not yet been explored deeply, however, since explainability is supposed to strengthen trust in artificial intelligence, it is necessary to systematically review and compare explanation methods in order to confirm their correctness. Until now, no tool with focus on XAI evaluation exists that exhaustively and speedily allows researchers to evaluate the performance of explanations of neural network predictions. To increase transparency and reproducibility in the field, we therefore built Quantus -- a comprehensive, evaluation toolkit in Python that includes a growing, well-organised collection of evaluation metrics and tutorials for evaluating explainable methods. The toolkit has been thoroughly tested and is available under an open-source license on PyPi (or on https://github.com/understandable-machine-intelligence-lab/Quantus/).

Key facts. Cited by 171 papers in the map (data through 14 Oct 2026). Builds on 40 of its 57 references that are in the map. First posted on arXiv 14 Feb 2022.

Builds on

40 of 57 references.

References · 40

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  6. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  7. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  8. 2018Sanity Checks for Saliency Mapscited by 1,156
  9. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 664
  10. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
  11. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  12. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 419
28 more
  1. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 365
  2. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 345
  3. 2017The (Un)reliability of saliency methodscited by 333
  4. 2016Top-down Neural Attention by Excitation Backpropcited by 321
  5. 2019On the (In)fidelity and Sensitivity for Explanationscited by 310
  6. 2018Metrics for Explainable AI: Challenges and Prospectscited by 260
  7. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 259
  8. 2020Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?cited by 190
  9. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 188
  10. 2019One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniquescited by 164
  11. 2020Debugging Tests for Model Explanationscited by 147
  12. 2020Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAIcited by 126
  13. 2019When Explanations Lie: Why Many Modified BP Attributions Failcited by 92
  14. 2019Benchmarking Attribution Methods with Relative Feature Importancecited by 89
  15. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 87
  16. 2020On quantitative aspects of model interpretabilitycited by 71
  17. 2019Towards Best Practice in Explaining Neural Network Decisions with LRPcited by 70
  18. 2019Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanationscited by 64
  19. 2018iNNvestigate neural networks!cited by 58
  20. 2022Framework for Evaluating Faithfulness of Local Explanationscited by 55
  21. 2020IROF: a low resource evaluation metric for explanation methodscited by 46
  22. 2020dalex: Responsible Machine Learning with Interactive Explainability and Fairness in Pythoncited by 40
  23. 2021Interpretable Semantic Photo Geolocationcited by 32
  24. 2021NoiseGrad: Enhancing Explanations by Introducing Stochasticity to Model Weightscited by 28
  25. 2021Focus! Rating XAI Methods and Finding Biasescited by 20
  26. 2021Revisiting Sanity Checks for Saliency Mapscited by 20
  27. 2021Explaining Bayesian Neural Networkscited by 18
  28. 2021Evaluating saliency methods on artificial data with different background typescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 13; 2023: 32; 2024: 60; 2025: 32; 2026: 34.

171 papers.

Sort:

By area:

  1. 2022OpenXAI: Towards a Transparent Evaluation of Model Explanationscited by 126
  2. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 125
  3. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 120
  4. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 73
  5. 2023FUTURE-AI: International consensus guideline for trustworthy and deployable artificial intelligence in healthcarecited by 60
  6. 2023FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methodscited by 50
  7. 2023Adversarial attacks and defenses in explainable artificial intelligence: A surveycited by 48
  8. 2024Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Directioncited by 46
  9. 2024shapiq: Shapley Interactions for Machine Learningcited by 43
  10. 2022Don't Lie to Me! Robust and Efficient Explainability with Verified Perturbation Analysiscited by 38
  11. 2023The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantuscited by 37
  12. 2022ProtoVAE: A Trustworthy Self-Explainable Prototypical Variational Modelcited by 34
38 more
  1. 2022A Comprehensive Survey on Trustworthy Recommender Systemscited by 32
  2. 2024Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarkscited by 28
  3. 2023Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Sciencecited by 27
  4. 2023Opti-CAM: Optimizing saliency maps for interpretabilitycited by 25
  5. 2023Explainable AI for Time Series via Virtual Inspection Layerscited by 23
  6. 2023Towards Evaluating Explanations of Vision Transformers for Medical Imagingcited by 23
  7. 2023Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanationscited by 22
  8. 2024Position: Explain to Question not to Justifycited by 20
  9. 2025Block-Recurrent Dynamics in Vision Transformerscited by 20
  10. 2022Revealing Hidden Context Bias in Segmentation and Object Detection through Concept-specific Explanationscited by 17
  11. 2024CoSy: Evaluating Textual Explanations of Neuronscited by 17
  12. 2024A comprehensive study on fidelity metrics for XAIcited by 16
  13. 2024Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metricscited by 16
  14. 2024Global Counterfactual Directionscited by 15
  15. 2025Mechanistic Interpretability as Statistical Estimation: A Variance Analysiscited by 15
  16. 2022DORA: Exploring Outlier Representations in Deep Neural Networkscited by 14
  17. 2023CLIP-QDA: An Explainable Concept Bottleneck Modelcited by 13
  18. 2024Listenable Maps for Audio Classifierscited by 13
  19. 2024Understanding Inter-Concept Relationships in Concept-Based Modelscited by 13
  20. 2023Explainability of Vision Transformers: A Comprehensive Review and New Perspectivescited by 12
  21. 2024Establishing and Evaluating Trustworthy AI: Overview and Research Challengescited by 12
  22. 2022Optimizing Explanations by Network Canonization and Hyperparameter Searchcited by 11
  23. 2024BEExAI: Benchmark to Evaluate Explainable AIcited by 11
  24. 2024Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformerscited by 11
  25. 2023Can We Trust Explainable AI Methods on ASR? An Evaluation on Phoneme Recognitioncited by 10
  26. 2023Why is plausibility surprisingly problematic as an XAI criterion?cited by 9
  27. 2024Sanity Checks Revisited: An Exploration to Repair the Model Parameter Randomisation Testcited by 9
  28. 2023Causal Analysis for Robust Interpretability of Neural Networkscited by 8
  29. 2024Explainable Bayesian Optimizationcited by 8
  30. 2024Rethinking Explainable Machine Learning as Applied Statisticscited by 8
  31. 2024A Fresh Look at Sanity Checks for Saliency Mapscited by 8
  32. 2024Benchmarking the Attribution Quality of Vision Modelscited by 8
  33. 2024Benchmarking XAI Explanations with Human-Aligned Evaluationscited by 8
  34. 2022SoK: Modeling Explainability in Security Analytics for Interpretability, Trustworthiness, and Usabilitycited by 7
  35. 2023XAI-TRIS: Non-linear image benchmarks to quantify false positive post-hoc attribution of feature importancecited by 7
  36. 2024Classification Metrics for Image Explanations: Towards Building Reliable XAI-Evaluationscited by 7
  37. 2025Towards Desiderata-Driven Design of Visual Counterfactual Explainerscited by 7
  38. 2023Saliency-Bench: A Comprehensive Benchmark for Evaluating Visual Explanationscited by 6

50 most cited of 171 All 171 (JSON)