mlsanityData to 7 Oct 2026

Position: Explain to Question not to Justify

arXiv 2402.13914 ↗First posted 21 Feb 2024cs.AI cs.CR cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

Explainable Artificial Intelligence (XAI) is a young but very promising field of research. Unfortunately, the progress in this field is currently slowed down by divergent and incompatible goals. We separate various threads tangled within the area of XAI into two complementary cultures of human/value-oriented explanations (BLUE XAI) and model/validation-oriented explanations (RED XAI). This position paper argues that the area of RED XAI is currently under-explored, i.e., more methods for explainability are desperately needed to question models (e.g., extract knowledge from well-performing models as well as spotting and fixing bugs in faulty models), and the area of RED XAI hides great opportunities and potential for important research necessary to ensure the safety of AI systems. We conclude this paper by presenting promising challenges in this area.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 23 of its 52 references that are in the map. First posted on arXiv 21 Feb 2024.

Builds on

23 of 52 references.

References · 23

  1. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  3. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  4. 2016The Mythos of Model Interpretabilitycited by 716
  5. 2018All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneouslycited by 364
  6. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
  7. 2019The What-If Tool: Interactive Probing of Machine Learning Modelscited by 207
  8. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  9. 2020Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAIcited by 125
  10. 2021Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunitiescited by 125
  11. 2021Acquisition of Chess Knowledge in AlphaZerocited by 120
  12. 2019On the Existence of Simpler Machine Learning Modelscited by 87
11 more
  1. 2022Exploring the Whole Rashomon Set of Sparse Decision Treescited by 80
  2. 2019explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learningcited by 69
  3. 2020One Explanation Does Not Fit All: The Promise of Interactive Explanations for Machine Learning Transparencycited by 64
  4. 2018iNNvestigate neural networks!cited by 58
  5. 2022Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvementcited by 48
  6. 2020The Grammar of Interactive Explanatory Model Analysiscited by 24
  7. 2023Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanationscited by 21
  8. 2020What Would You Ask the Machine Learning Model? Identification of User Needs for Model Explanations Based on Human-Model Conversationscited by 11
  9. 2023A New Perspective on Evaluation Methods for Explainable Artificial Intelligence (XAI)cited by 5
  10. 2023Be Careful When Evaluating Explanations Regarding Ground Truthcited by 3
  11. 2021i-Algebra: Towards Interactive Interpretability of Deep Neural Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 9; 2026: 5.

20 papers.

Sort:

  1. 2025Interpreting CLIP with Hierarchical Sparse Autoencoderscited by 54
  2. 2025Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learningcited by 17
  3. 2024Rethinking Visual Counterfactual Explanations Through Region Constraintcited by 14
  4. 2024GraphXAIN: Narratives to Explain Graph Neural Networkscited by 11
  5. 2024Controllable Continual Test-Time Adaptationcited by 6
  6. 2025Beyond the Single-Best Model: Rashomon Partial Dependence Profile for Trustworthy Explanations in AutoMLcited by 6
  7. 2024Aggregated Attributions for Explanatory Analysis of 3D Segmentation Modelscited by 5
  8. 2024X-SHIELD: Regularization for eXplainable Artificial Intelligencecited by 3
  9. 2025Birds look like cars: Adversarial analysis of intrinsically interpretable deep learningcited by 3
  10. 2025Model Science: getting serious about verification, explanation and control of AI systemscited by 3
  11. 2025STOOD-X methodology: using statistical nonparametric test for OOD Detection Large-Scale datasets enhanced with explainabilitycited by 2
  12. 2025A three-Level Framework for LLM-Enhanced eXplainable AI: From technical explanations to natural languagecited by 2
8 more
  1. 2026Auditing Sybil: Explaining Deep Lung Cancer Risk Prediction Through Generative Interventional Attributionscited by 2
  2. 2026The Case for Model Science: Verify, Explore, Steer, Refinecited by 2
  3. 2025Predictive Multiplicity in Survival Models: A Method for Quantifying Model Uncertainty in Predictive Maintenance Applicationscited by 1
  4. 2026Exploring SAIG Methods for an Objective Evaluation of XAIcited by 1
  5. 2024RAB2-DEF: Dynamic and explainable defense against adversarial attacks in Federated Learning to fair poor clientscited by 0
  6. 2025NeuroFaith: Evaluating LLM Self-Explanation Faithfulness via Internal Representation Alignmentcited by 0
  7. 2026Analyzing the Temporal Factors for Anxiety and Depression Symptoms with the Rashomon Perspectivecited by 0
  8. 2026Selecting The Most Informative Tokens in Natural Language Autoencoderscited by 0