mlsanityData to 7 Oct 2026

FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methods

arXiv 2308.06248 ↗First posted 11 Aug 2023cs.CV cs.LGcited by 49

Not on the map. Why?

Abstract (arXiv)

The field of explainable artificial intelligence (XAI) aims to uncover the inner workings of complex deep neural models. While being crucial for safety-critical domains, XAI inherently lacks ground-truth explanations, making its automatic evaluation an unsolved problem. We address this challenge by proposing a novel synthetic vision dataset, named FunnyBirds, and accompanying automatic evaluation protocols. Our dataset allows performing semantically meaningful image interventions, e.g., removing individual object parts, which has three important implications. First, it enables analyzing explanations on a part level, which is closer to human comprehension than existing methods that evaluate on a pixel level. Second, by comparing the model output for inputs with removed parts, we can estimate ground-truth part importances that should be reflected in the explanations. Third, by mapping individual explanations into a common space of part importances, we can analyze a variety of different explanation types in a single common framework. Using our tools, we report results for 24 different combinations of neural models and XAI methods, demonstrating the strengths and weaknesses of the assessed methods in a fully automatic and systematic manner.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 51 of its 86 references that are in the map. First posted on arXiv 11 Aug 2023.

Builds on

51 of 86 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  6. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  7. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  9. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  10. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  11. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  12. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
38 more
  1. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  2. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  3. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  4. 2020Quantifying Attention Flow in Transformerscited by 759
  5. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  6. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  7. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  8. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  9. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
  10. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  11. 2016Top-down Neural Attention by Excitation Backpropcited by 318
  12. 2019Counterfactual Visual Explanationscited by 312
  13. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  14. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 295
  15. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 293
  16. 2018Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negativescited by 261
  17. 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
  18. 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
  19. 2016Understanding Neural Networks through Representation Erasurecited by 223
  20. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 213
  21. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  22. 2017Visualizing Deep Neural Network Decisions: Prediction Difference Analysiscited by 166
  23. 2016Generating Visual Explanationscited by 156
  24. 2020Debugging Tests for Model Explanationscited by 146
  25. 2018Explaining Image Classifiers by Counterfactual Generationcited by 144
  26. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  27. 2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
  28. 2018Interpreting CNNs via Decision Treescited by 89
  29. 2022B-cos Networks: Alignment is All We Need for Interpretabilitycited by 88
  30. 2021The effectiveness of feature attribution methods and its correlation with automatic evaluation scorescited by 87
  31. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 86
  32. 2021Convolutional Dynamic Alignment Networks for Interpretable Classificationscited by 54
  33. 2020Generative causal explanations of black-box classifierscited by 41
  34. 2020How Useful Are the Machine-Generated Interpretations to General Users? A Human Evaluation on Guessing the Incorrectly Predicted Labelscited by 39
  35. 2021Fast Axiomatic Attribution for Neural Networkscited by 35
  36. 2018Towards Explanation of DNN-based Prediction with Guided Feature Inversioncited by 31
  37. 2017Visual Explanation by Interpretation: Improving Visual Feedback Capabilities of Deep Neural Networkscited by 28
  38. 2022Towards Better Understanding Attribution Methodscited by 28

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 17; 2025: 19; 2026: 10.

49 papers.

Sort:

  1. 2024LeGrad: An Explainability Method for Vision Transformers via Feature Formation Sensitivitycited by 32
  2. 2023Diversify, Don't Fine-Tune: Scaling Up Visual Recognition Training with Synthetic Imagescited by 20
  3. 2023Pixel-Grounded Prototypical Part Networkscited by 18
  4. 2024Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metricscited by 15
  5. 2024ProtoS-ViT: Visual foundation models for sparse self-explainable classificationscited by 8
  6. 2024Benchmarking the Attribution Quality of Vision Modelscited by 7
  7. 2024FungiTastic: A multi-modal dataset and benchmark for image categorizationcited by 7
  8. 2024InfoDisent: Explainability of Image Classification Models by Information Disentanglementcited by 7
  9. 2023Saliency-Bench: A Comprehensive Benchmark for Evaluating Visual Explanationscited by 6
  10. 2025EPIC: Explanation of Pretrained Image Classification Networks via Prototypecited by 6
  11. 2025Comprehensive Information Bottleneck for Unveiling Universal Attribution to Interpret Vision Transformerscited by 6
  12. 2025SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutionscited by 6
37 more
  1. 2024Reactive Model Correction: Mitigating Harm to Task-Relevant Features via Conditional Bias Suppressioncited by 5
  2. 2024On the Evaluation Consistency of Attribution-based Explanationscited by 5
  3. 2024LibraGrad: Balancing Gradient Flow for Universally Better Vision Transformer Attributionscited by 5
  4. 2024Arctique: An artificial histopathological dataset unifying realism and controllability for uncertainty quantificationcited by 4
  5. 2024This Probably Looks Exactly Like That: An Invertible Prototypical Networkcited by 3
  6. 2025This looks like what? Challenges and Future Research Directions for Part-Prototype Modelscited by 3
  7. 2025Birds look like cars: Adversarial analysis of intrinsically interpretable deep learningcited by 3
  8. 2025SIDE: Sparse Information Disentanglement for Explainable Artificial Intelligencecited by 3
  9. 2024ComFe: An Interpretable Head for Vision Transformerscited by 2
  10. 2024Revisiting FunnyBirds evaluation framework for prototypical parts networkscited by 2
  11. 2024OMENN: One Matrix to Explain Neural Networkscited by 2
  12. 2025Tell me why: Visual foundation models as self-explainable classifierscited by 2
  13. 2025Locally Explaining Prediction Behavior via Gradual Interventions and Measuring Property Gradientscited by 2
  14. 2025Personalized Interpretability -- Interactive Alignment of Prototypical Parts Networkscited by 2
  15. 2025FunnyNodules: A Customizable Medical Dataset Tailored for Evaluating Explainable AIcited by 2
  16. 2024Data Science Principles for Interpretable and Explainable AIcited by 1
  17. 2024Inpainting the Gaps: A Novel Framework for Evaluating Explanation Methods in Vision Transformerscited by 1
  18. 2025COMIX: Compositional Explanations using Prototypescited by 1
  19. 2025Post-Hoc Concept Disentanglement: From Correlated to Isolated Concept Representationscited by 1
  20. 2025Beyond Accuracy: What Matters in Designing Well-Behaved Image Classification Models?cited by 1
  21. 2025FaCT: Faithful Concept Traces for Explaining Neural Network Decisionscited by 1
  22. 2025Measuring the Effect of Background on Classification and Feature Importance in Deep Learning for AV Perceptioncited by 1
  23. 2026Exploring SAIG Methods for an Objective Evaluation of XAIcited by 1
  24. 2026What is Missing? Explaining Neurons Activated by Absent Conceptscited by 1
  25. 2024PCEvE: Part Contribution Evaluation Based Model Explanation for Human Figure Drawing Assessment and Beyondcited by 0
  26. 2025SIC: Similarity-Based Interpretable Image Classification with Neural Networkscited by 0
  27. 2025BlueGlass: A Framework for Composite AI Safetycited by 0
  28. 2025Value bounds and Convergence Analysis for Averages of LRP attributionscited by 0
  29. 2025CAMBench-QR : A Structure-Aware Benchmark for Post-Hoc Explanations with QR Understandingcited by 0
  30. 2026ProtoQuant: Quantization of Prototypical Parts For General and Fine-Grained Image Classificationcited by 0
  31. 2026Missingness Bias Calibration in Feature Attribution Explanationscited by 0
  32. 2026How to Evaluate and Refine your CAMcited by 0
  33. 2026Does Explainability Transfer? A Controlled Benchmark of Attribution Methods on Vision Transformers and CNNscited by 0
  34. 2026Grad-CAM for Vision Transformers: A Systematic Taxonomy and Audit of Methodological Ambiguity in Explainable AIcited by 0
  35. 2026Decomposition of Evidence, Contradiction, and Fragility in Perturbation Responsescited by 0
  36. 2026HiLRP: Toward One Trustworthy Explanation for Vision Transformer: Conservation-Valid Attribution via Attention Primitivescited by 0
  37. 2026ResLRP: The Role of Residual Cancellation in Attribution Instability in Vision Transformerscited by 0