mlsanityData to 7 Oct 2026

Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions

arXiv 2310.19775 ↗First posted 30 Oct 2023cs.AIcited by 123

Not on the map. Why?

Abstract (arXiv)

As systems based on opaque Artificial Intelligence (AI) continue to flourish in diverse real-world applications, understanding these black box models has become paramount. In response, Explainable AI (XAI) has emerged as a field of research with practical and ethical benefits across various domains. This paper not only highlights the advancements in XAI and its application in real-world scenarios but also addresses the ongoing challenges within XAI, emphasizing the need for broader perspectives and collaborative efforts. We bring together experts from diverse fields to identify open problems, striving to synchronize research agendas and accelerate XAI in practical applications. By fostering collaborative discussion and interdisciplinary cooperation, we aim to propel XAI forward, contributing to its continued success. Our goal is to put forward a comprehensive proposal for advancing XAI. To achieve this goal, we present a manifesto of 27 open problems categorized into nine categories. These challenges encapsulate the complexities and nuances of XAI and offer a road map for future research. For each problem, we provide promising research directions in the hope of harnessing the collective intelligence of interested stakeholders.

Key facts. Cited by 123 papers in the map (data through 7 Oct 2026). Builds on 126 of its 304 references that are in the map. First posted on arXiv 30 Oct 2023.

Builds on

126 of 304 references.

References · 126

  1. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  3. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  4. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  5. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  6. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  7. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  8. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  9. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  10. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  11. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  12. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
38 more
  1. 2020Concept Bottleneck Modelscited by 1,022
  2. 2019Machine Unlearningcited by 1,020
  3. 2022Diffusion Models in Vision: A Surveycited by 1,008
  4. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  5. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  6. 2020Quantifying Attention Flow in Transformerscited by 759
  7. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  8. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  9. 2023A Watermark for Large Language Modelscited by 667
  10. 2019TabNet: Attentive Interpretable Tabular Learningcited by 640
  11. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
  12. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 440
  13. 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 403
  14. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 398
  15. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 396
  16. 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 369
  17. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 359
  18. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  19. 2019Interpretable machine learning: definitions, methods, and applicationscited by 303
  20. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  21. 2022A Survey of Machine Unlearningcited by 298
  22. 2018Metrics for Explainable AI: Challenges and Prospectscited by 257
  23. 2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 244
  24. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 213
  25. 2020Logic Tensor Networkscited by 197
  26. 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
  27. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  28. 2019One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniquescited by 164
  29. 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 153
  30. 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
  31. 2018Explaining Explanations in AIcited by 136
  32. 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
  33. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  34. 2020Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAIcited by 125
  35. 2022OpenXAI: Towards a Transparent Evaluation of Model Explanationscited by 125
  36. 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
  37. 2019Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approachescited by 101
  38. 2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100

50 most cited of 126 All 126 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 43; 2025: 37; 2026: 42.

123 papers.

Sort:

By area:

  1. 2024Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Directioncited by 46
  2. 2025LLMs for Explainable AI: A Comprehensive Surveycited by 42
  3. 2025Mechanistic understanding and validation of large AI models with SemanticLenscited by 29
  4. 2024Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agendacited by 22
  5. 2024Global Counterfactual Directionscited by 15
  6. 2025An overview of model uncertainty and variability in LLM-based sentiment analysis. Challenges, mitigation strategies and the role of explainabilitycited by 15
  7. 2024BEExAI: Benchmark to Evaluate Explainable AIcited by 11
  8. 2024CheckEmbed: Effective Verification of LLM Solutions to Open-Ended Taskscited by 10
  9. 2025Winsor-CAM: Human-Tunable Visual Explanations from Deep Networks via Layer-Wise Winsorizationcited by 10
  10. 2025RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patchingcited by 10
  11. 2024Interpretable Clustering: A Surveycited by 8
  12. 2024Towards Unifying Evaluation of Counterfactual Explanations: Leveraging Large Language Models for Human-Centric Assessmentscited by 8
38 more
  1. 2023Local Concept Embeddings for Analysis of Concept Distributions in Vision DNN Feature Spacescited by 7
  2. 2025Is Trust Correlated With Explainability in AI? A Meta-Analysiscited by 7
  3. 2024What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blockscited by 6
  4. 2024"Oh LLM, I'm Asking Thee, Please Give Me a Decision Tree": Zero-Shot Decision Tree Induction and Embedding with Large Language Modelscited by 6
  5. 2025Towards Achieving Concept Completeness for Textual Concept Bottleneck Modelscited by 6
  6. 2025Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networkscited by 6
  7. 2024Triamese-ViT: A 3D-Aware Method for Robust Brain Age Estimation from MRIscited by 5
  8. 2025A Framework for Responsible AI Systems: Building Societal Trust through Domain Definition, Trustworthy AI Design, Auditability, Accountability, and Governancecited by 5
  9. 2024Large Language Models Meet User Interfaces: The Case of Provisioning Feedbackcited by 4
  10. 2024Exploring Human-LLM Conversations: Mental Models and the Originator of Toxicitycited by 4
  11. 2024A Taxonomy of Architecture Options for Foundation Model-based Agents: Analysis and Decision Modelcited by 4
  12. 2024User-centric evaluation of explainability of AI with and for humans: a comprehensive empirical studycited by 4
  13. 2024Learning New Concepts, Remembering the Old: Continual Learning for Multimodal Concept Bottleneck Modelscited by 4
  14. 2024Manipulating Feature Visualizations with Gradient Slingshotscited by 3
  15. 2024The Effect of Data Poisoning on Counterfactual Explanationscited by 3
  16. 2024X-SHIELD: Regularization for eXplainable Artificial Intelligencecited by 3
  17. 2024GPT Assisted Annotation of Rhetorical and Linguistic Features for Interpretable Propaganda Technique Detection in News Textcited by 3
  18. 2025Gender Bias in Explainability: Investigating Performance Disparity in Post-hoc Methodscited by 3
  19. 2025AI-Driven Automation Can Become the Foundation of Next-Era Science of Science Researchcited by 3
  20. 2025Generative Models for Crystalline Materialscited by 3
  21. 2024Governance of Generative Artificial Intelligence for Companiescited by 2
  22. 2024Knowledge graphs for empirical concept retrievalcited by 2
  23. 2024Solving the enigma: Enhancing faithfulness and comprehensibility in explanations of deep networkscited by 2
  24. 2024Deciphering the Definition of Adversarial Robustness for post-hoc OOD Detectorscited by 2
  25. 2024CiteFusion: An Ensemble Framework for Citation Intent Classification Harnessing Dual-Model Binary Couples and SHAP Analysescited by 2
  26. 2024Patch-based Intuitive Multimodal Prototypes Network (PIMPNet) for Alzheimer's Disease classificationcited by 2
  27. 2024Topic Modeling with Fine-tuning LLMs and Bag of Sentencescited by 2
  28. 2024Understanding with toy surrogate models in machine learningcited by 2
  29. 2024An Open API Architecture to Discover the Trustworthy Explanation of Cloud AI Servicescited by 2
  30. 2025How Metacognitive Architectures Remember Their Own Thoughts: A Systematic Reviewcited by 2
  31. 2025STOOD-X methodology: using statistical nonparametric test for OOD Detection Large-Scale datasets enhanced with explainabilitycited by 2
  32. 2025See What I Mean? CUE: A Cognitive Model of Understanding Explanationscited by 2
  33. 2025Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanationscited by 2
  34. 2025Rethinking Saliency Maps: A Cognitive Human Aligned Taxonomy and Evaluation Framework for Explanationscited by 2
  35. 2026In Search of Grandmother Cells: Tracing Interpretable Neurons in Tabular Representationscited by 2
  36. 2026The Case for Model Science: Verify, Explore, Steer, Refinecited by 2
  37. 2024CNN-based explanation ensembling for dataset, representation and explanations evaluationcited by 1
  38. 2024Rad4XCNN: a new agnostic method for post-hoc global explanation of CNN-derived features by means of radiomicscited by 1

50 most cited of 123 All 123 (JSON)