# Explainable artificial intelligence (XAI): from inherent explainability to large language models

arXiv: https://arxiv.org/abs/2501.09967 · HTML: https://mlsanity.com/paper/2501.09967/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jan 2025.

## Built on by (most cited first, 12 of 12)

- [Layered Chain-of-Thought Prompting for Multi-Agent LLM Systems: A Comprehensive Approach to Explainable Large Language Models](https://mlsanity.com/paper/2501.18645/) (2025)
- [Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency maps](https://mlsanity.com/paper/2504.17023/) (2025)
- [Understanding the Dilemma of Unlearning for Large Language Models](https://mlsanity.com/paper/2509.24675/) (2025)
- [ODExAI: A Comprehensive Object Detection Explainable AI Evaluation](https://mlsanity.com/paper/2504.19249/) (2025)
- [Glaucoma Detection and Structured OCT Report Generation via a Fine-tuned Multimodal Large Language Model](https://mlsanity.com/paper/2510.02403/) (2025)
- [Explainable AI for Mental Health Emergency Returns: Integrating LLMs with Predictive Modeling](https://mlsanity.com/paper/2502.00025/) (2025)
- [Triadic Fusion of Cognitive, Functional, and Causal Dimensions for Explainable LLMs: The TAXAL Framework](https://mlsanity.com/paper/2509.05199/) (2025)
- [Deliberative Reasoning Network: An Uncertainty-Driven Paradigm for Belief-Tracked Inference with Pretrained Language Models](https://mlsanity.com/paper/2508.04339/) (2025)
- [Beyond Confidence: The Rhythms of Reasoning in Generative Models](https://mlsanity.com/paper/2602.10816/) (2026)
- [The Confusion is Real: GRAPHIC -- A Network Science Approach to Confusion Matrices in Deep Learning](https://mlsanity.com/paper/2602.19770/) (2026)
- [XtrAIn: Training-Guided Occlusion for Feature Attribution](https://mlsanity.com/paper/2606.10877/) (2026)
- [REALM: Regime-Switching, Explainable, and Activation-Induced Linear Models](https://mlsanity.com/paper/2609.32141/) (2026)

## Abstract (arXiv)

Artificial Intelligence (AI) has continued to achieve tremendous success in recent times. However, the decision logic of these frameworks is often not transparent, making it difficult for stakeholders to understand, interpret or explain their behavior. This limitation hinders trust in machine learning systems and causes a general reluctance towards their adoption in practical applications, particularly in mission-critical domains like healthcare and autonomous driving. Explainable AI (XAI) techniques facilitate the explainability or interpretability of machine learning models, enabling users to discern the basis of the decision and possibly avert undesirable behavior. This comprehensive survey details the advancements of explainable AI methods, from inherently interpretable models to modern approaches for achieving interpretability of various black box models, including large language models (LLMs). Additionally, we review explainable AI techniques that leverage LLM and vision-language model (VLM) frameworks to automate or improve the explainability of other machine learning models. The use of LLM and VLM as interpretability methods particularly enables high-level, semantically meaningful explanations of model decisions and behavior. Throughout the paper, we highlight the scientific principles, strengths and weaknesses of state-of-the-art methods and outline different areas of improvement. Where appropriate, we also present qualitative and quantitative comparison results of various methods to show how they compare. Finally, we discuss the key challenges of XAI and directions for future research.