XAI Handbook: Towards a Unified Framework for Explainable AI
Not on the map. Why?
Abstract (arXiv)
The field of explainable AI (XAI) has quickly become a thriving and prolific community. However, a silent, recurrent and acknowledged issue in this area is the lack of consensus regarding its terminology. In particular, each new contribution seems to rely on its own (and often intuitive) version of terms like "explanation" and "interpretation". Such disarray encumbers the consolidation of advances in the field towards the fulfillment of scientific and regulatory demands e.g., when comparing methods or establishing their compliance with respect to biases and fairness constraints. We propose a theoretical framework that not only provides concrete definitions for these terms, but it also outlines all steps necessary to produce explanations and interpretations. The framework also allows for existing contributions to be re-contextualized such that their scope can be measured, thus making them comparable to other methods. We show that this framework is compliant with desiderata on explanations, on interpretability and on evaluation metrics. We present a use-case showing how the framework can be used to compare LIME, SHAP and MDNet, establishing their advantages and shortcomings. Finally, we discuss relevant trends in XAI as well as recommendations for future work, all from the standpoint of our framework.
Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 35 of its 60 references that are in the map. First posted on arXiv 14 May 2021.
Builds on
35 of 60 references.
References · 35
- 2017Attention Is All You Needcited by 60,361
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2013Intriguing properties of neural networkscited by 5,155
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
- 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
- 2019Attention is not Explanationcited by 929
- 2016The Mythos of Model Interpretabilitycited by 721
23 more
- 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
- 2019Attention is not not Explanationcited by 636
- 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
- 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 496
- 2019On Evaluating Adversarial Robustnesscited by 447
- 2017Interpretation of Neural Networks is Fragilecited by 444
- 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 443
- 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 345
- 2019Explanations can be manipulated and geometry is to blamecited by 216
- 2018Explaining Image Classifiers by Counterfactual Generationcited by 144
- 2020Explainable Deep Learning: A Field Guide for the Uninitiatedcited by 127
- 2019Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approachescited by 101
- 2017Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciencescited by 85
- 2016Discovering Causal Signals in Imagescited by 79
- 2020Explainable Artificial Intelligence: a Systematic Reviewcited by 79
- 2020A Disentangling Invertible Interpretation Network for Explaining Latent Representationscited by 57
- 2017MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Networkcited by 56
- 2020Towards falsifiable interpretability researchcited by 53
- 2018Explanation Methods in Deep Learning: Users, Values, Concerns and Challengescited by 49
- 2017Contextual Explanation Networkscited by 21
- 2018Explainable Software Analyticscited by 15
- 2018What do Deep Networks Like to See?cited by 4
- 2020Evolved Explainable Classifications for Lymph Node Metastasescited by 2
Built on by
Citing papers per year: 2021: 1; 2022: 1; 2023: 4; 2024: 4; 2025: 4.
14 papers.
Sort:
By area:
- 2025Interpretable Machine Learning in Physics: A Reviewcited by 14
- 2021Explainability Is in the Mind of the Beholder: Establishing the Foundations of Explainable Artificial Intelligencecited by 13
- 2022Why Should I Choose You? AutoXAI: A Framework for Selecting and Tuning eXplainable AI Solutionscited by 10
- 2024Generating Counterfactual Trajectories with Latent Diffusion Models for Concept Discoverycited by 8
- 2023Categorical Foundations of Explainable AI: A Unifying Theorycited by 7
- 2025Mathematical Foundation of Interpretable Equivariant Surrogate Modelscited by 5
- 2023A Co-design Study for Multi-Stakeholder Job Recommender System Explanationscited by 4
- 2025On the Design and Evaluation of Human-centered Explainable AI Systems: A Systematic Review and Taxonomycited by 3
- 2024An Open API Architecture to Discover the Trustworthy Explanation of Cloud AI Servicescited by 2
- 2024Cloud-based XAI Services for Assessing Open Repository Models Under Adversarial Attackscited by 1
- 2025Logic Explanation of AI Classifiers by Categorical Explaining Functorscited by 1
- 2023AI2: The next leap toward native language based and explainable machine learning frameworkcited by 0
- 2025On the Design and Evaluation of Human-centered Explainable AI Systems: A Systematic Review and Taxonomy14 Oct 2025
- 2025Interpretable Machine Learning in Physics: A Review30 Mar 2025
- 2025Logic Explanation of AI Classifiers by Categorical Explaining Functors20 Mar 2025
- 2025Mathematical Foundation of Interpretable Equivariant Surrogate Models3 Mar 2025
- 2024An Open API Architecture to Discover the Trustworthy Explanation of Cloud AI Services5 Nov 2024
- 2024Explainable AI: Definition and attributes of a good explanation for health AI9 Sep 2024
- 2024Generating Counterfactual Trajectories with Latent Diffusion Models for Concept Discovery16 Apr 2024
- 2024Cloud-based XAI Services for Assessing Open Repository Models Under Adversarial Attacks22 Jan 2024
- 2023Extracting Interpretable Local and Global Representations from Attention on Time Series16 Sep 2023
- 2023A Co-design Study for Multi-Stakeholder Job Recommender System Explanations11 Sep 2023
- 2023Categorical Foundations of Explainable AI: A Unifying Theory27 Apr 2023
- 2023AI2: The next leap toward native language based and explainable machine learning framework9 Jan 2023
- 2025Interpretable Machine Learning in Physics: A Reviewcited by 14
- 2021Explainability Is in the Mind of the Beholder: Establishing the Foundations of Explainable Artificial Intelligencecited by 13
- 2022Why Should I Choose You? AutoXAI: A Framework for Selecting and Tuning eXplainable AI Solutionscited by 10
- 2023Categorical Foundations of Explainable AI: A Unifying Theorycited by 7
- 2025Mathematical Foundation of Interpretable Equivariant Surrogate Modelscited by 5
- 2023A Co-design Study for Multi-Stakeholder Job Recommender System Explanationscited by 4
- 2025On the Design and Evaluation of Human-centered Explainable AI Systems: A Systematic Review and Taxonomycited by 3
- 2024An Open API Architecture to Discover the Trustworthy Explanation of Cloud AI Servicescited by 2
- 2024Cloud-based XAI Services for Assessing Open Repository Models Under Adversarial Attackscited by 1
- 2025Logic Explanation of AI Classifiers by Categorical Explaining Functorscited by 1
- 2023AI2: The next leap toward native language based and explainable machine learning frameworkcited by 0
- 2023Extracting Interpretable Local and Global Representations from Attention on Time Seriescited by 0
12 most cited of 13 All 13 (JSON)
- 2025On the Design and Evaluation of Human-centered Explainable AI Systems: A Systematic Review and Taxonomy14 Oct 2025
- 2025Interpretable Machine Learning in Physics: A Review30 Mar 2025
- 2025Logic Explanation of AI Classifiers by Categorical Explaining Functors20 Mar 2025
- 2025Mathematical Foundation of Interpretable Equivariant Surrogate Models3 Mar 2025
- 2024An Open API Architecture to Discover the Trustworthy Explanation of Cloud AI Services5 Nov 2024
- 2024Explainable AI: Definition and attributes of a good explanation for health AI9 Sep 2024
- 2024Cloud-based XAI Services for Assessing Open Repository Models Under Adversarial Attacks22 Jan 2024
- 2023Extracting Interpretable Local and Global Representations from Attention on Time Series16 Sep 2023
- 2023A Co-design Study for Multi-Stakeholder Job Recommender System Explanations11 Sep 2023
- 2023Categorical Foundations of Explainable AI: A Unifying Theory27 Apr 2023
- 2023AI2: The next leap toward native language based and explainable machine learning framework9 Jan 2023
- 2022Why Should I Choose You? AutoXAI: A Framework for Selecting and Tuning eXplainable AI Solutions6 Oct 2022
12 newest of 13 All 13 (JSON)