# The future of human-centric eXplainable Artificial Intelligence (XAI) is not post-hoc explanations

arXiv: https://arxiv.org/abs/2307.00364 · HTML: https://mlsanity.com/paper/2307.00364/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 53 references that are in the map.
- First posted on arXiv 1 Jul 2023.

## Builds on (most cited first, 26 of 26)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [Neural Additive Models: Interpretable Machine Learning with Neural Nets](https://mlsanity.com/paper/2004.13912/) (2020)
- [Counterfactual Visual Explanations](https://mlsanity.com/paper/1904.07451/) (2019)
- [Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives](https://mlsanity.com/paper/1802.07623/) (2018)
- [The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01602/) (2022)
- [Learning Model-Agnostic Counterfactual Explanations for Tabular Data](https://mlsanity.com/paper/1910.09398/) (2019)
- [Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems](https://mlsanity.com/paper/1907.09615/) (2019)
- [OpenXAI: Towards a Transparent Evaluation of Model Explanations](https://mlsanity.com/paper/2206.11104/) (2022)
- [Probing Across Time: What Does RoBERTa Know and When?](https://mlsanity.com/paper/2104.07885/) (2021)
- [Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations](https://mlsanity.com/paper/2205.07277/) (2022)
- [On Interpretability of Deep Learning based Skin Lesion Classifiers using Concept Activation Vectors](https://mlsanity.com/paper/2005.02000/) (2020)
- [Explainable Artificial Intelligence (XAI) from a user perspective- A synthesis of prior literature and problematizing avenues for future research](https://mlsanity.com/paper/2211.15343/) (2022)
- [AI in Education needs interpretable machine learning: Lessons from Open Learner Modelling](https://mlsanity.com/paper/1807.00154/) (2018)
- [Evaluating the Explainers: Black-Box Explainable Machine Learning for Student Success Prediction in MOOCs](https://mlsanity.com/paper/2207.00551/) (2022)
- [Interpreting Language Models Through Knowledge Graph Extraction](https://mlsanity.com/paper/2111.08546/) (2021)
- [Towards Interpretable Deep Learning Models for Knowledge Tracing](https://mlsanity.com/paper/2005.06139/) (2020)
- [RIPPLE: Concept-Based Interpretation for Raw Time Series Models in Education](https://mlsanity.com/paper/2212.01133/) (2022)
- [Trusting the Explainers: Teacher Validation of Explainable Artificial Intelligence for Course Design](https://mlsanity.com/paper/2212.08955/) (2022)
- [Disagreement amongst counterfactual explanations: How transparency can be deceptive](https://mlsanity.com/paper/2304.12667/) (2023)

## Built on by (most cited first, 12 of 12)

- [Intrinsic User-Centric Interpretability through Global Mixture of Experts](https://mlsanity.com/paper/2402.02933/) (2024)
- [Listening with Attention: Entropy-Guided Explainability for Transformer-Based Audio Models](https://mlsanity.com/paper/2606.14647/) (2026)
- [How Well Do Feature-Additive Explainers Explain Feature-Additive Predictors?](https://mlsanity.com/paper/2310.18496/) (2023)
- [Justifiable Artificial Intelligence: Engineering Large Language Models for Legal Applications](https://mlsanity.com/paper/2311.15716/) (2023)
- [Towards a Unified Framework for Evaluating Explanations](https://mlsanity.com/paper/2405.14016/) (2024)
- [Deep Learning for Educational Data Science](https://mlsanity.com/paper/2404.19675/) (2024)
- [Interpret3C: Interpretable Student Clustering Through Individualized Feature Selection](https://mlsanity.com/paper/2407.11979/) (2024)
- [In defence of post-hoc explanations in medical AI](https://mlsanity.com/paper/2504.20741/) (2025)
- [PRECISe : Prototype-Reservation for Explainable Classification under Imbalanced and Scarce-Data Settings](https://mlsanity.com/paper/2408.05754/) (2024)
- [A constraints-based approach to fully interpretable neural networks for detecting learner behaviors](https://mlsanity.com/paper/2504.20055/) (2025)
- [Mind the XAI Gap: A Human-Centered LLM Framework for Democratizing Explainable AI](https://mlsanity.com/paper/2506.12240/) (2025)
- [The Gold in Bias: Maturing the AI Design Process through Verification](https://mlsanity.com/paper/2609.29730/) (2026)

## Abstract (arXiv)

Explainable Artificial Intelligence (XAI) plays a crucial role in enabling human understanding and trust in deep learning systems. As models get larger, more ubiquitous, and pervasive in aspects of daily life, explainability is necessary to minimize adverse effects of model mistakes. Unfortunately, current approaches in human-centric XAI (e.g. predictive tasks in healthcare, education, or personalized ads) tend to rely on a single post-hoc explainer, whereas recent work has identified systematic disagreement between post-hoc explainers when applied to the same instances of underlying black-box models. In this paper, we therefore present a call for action to address the limitations of current state-of-the-art explainers. We propose a shift from post-hoc explainability to designing interpretable neural network architectures. We identify five needs of human-centric XAI (real-time, accurate, actionable, human-interpretable, and consistent) and propose two schemes for interpretable-by-design neural network workflows (adaptive routing with InterpretCC and temporal diagnostics with I2MD). We postulate that the future of human-centric XAI is neither in explaining black-boxes nor in reverting to traditional, interpretable models, but in neural networks that are intrinsically interpretable.