# Faithful and Plausible Natural Language Explanations for Image Classification: A Pipeline Approach

arXiv: https://arxiv.org/abs/2407.20899 · HTML: https://mlsanity.com/paper/2407.20899/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 36 references that are in the map.
- First posted on arXiv 30 Jul 2024.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [Textual Explanations for Self-Driving Vehicles](https://mlsanity.com/paper/1807.11546/) (2018)
- [Measuring Association Between Labels and Free-Text Rationales](https://mlsanity.com/paper/2010.12762/) (2020)
- [Generating Visual Explanations](https://mlsanity.com/paper/1603.08507/) (2016)
- [Natural Language Descriptions of Deep Visual Features](https://mlsanity.com/paper/2201.11114/) (2022)
- [Faithfulness Tests for Natural Language Explanations](https://mlsanity.com/paper/2305.18029/) (2023)
- [Faithful Multimodal Explanation for Visual Question Answering](https://mlsanity.com/paper/1809.02805/) (2018)
- [NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Tasks](https://mlsanity.com/paper/2203.05081/) (2022)
- [Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs](https://mlsanity.com/paper/2010.07526/) (2020)
- [Explaining Chest X-ray Pathologies in Natural Language](https://mlsanity.com/paper/2207.04343/) (2022)
- [Comprehensible Convolutional Neural Networks via Guided Concept Learning](https://mlsanity.com/paper/2101.03919/) (2021)
- [The Problem of Coherence in Natural Language Explanations of Recommendations](https://mlsanity.com/paper/2312.11356/) (2023)

## Built on by (most cited first, 7 of 7)

- [Counterfactual Simulatability of LLM Explanations for Generation Tasks](https://mlsanity.com/paper/2505.21740/) (2025)
- [An Agentic Approach to Generating XAI-Narratives](https://mlsanity.com/paper/2603.20003/) (2026)
- [Do LLMs Align Human Values Regarding Social Biases? Judging and Explaining Social Biases with LLMs](https://mlsanity.com/paper/2509.13869/) (2025)
- [Describe-Then-Act: Proactive Agent Steering via Distilled Language-Action World Models](https://mlsanity.com/paper/2603.23149/) (2026)
- [A Framework for Evaluating Faithfulness in Explainable AI for Machine Anomalous Sound Detection Using Frequency-Band Perturbation](https://mlsanity.com/paper/2601.19017/) (2026)
- [WeNLEX: Weakly Supervised Natural Language Explanations for Multilabel Chest X-ray Classification](https://mlsanity.com/paper/2603.18752/) (2026)
- [Zero-Shot Faithful Textual Explanations via Directional-Derivative Influence on Predictions](https://mlsanity.com/paper/2605.16877/) (2026)

## Abstract (arXiv)

Existing explanation methods for image classification struggle to provide faithful and plausible explanations. This paper addresses this issue by proposing a post-hoc natural language explanation method that can be applied to any CNN-based classifier without altering its training process or affecting predictive performance. By analysing influential neurons and the corresponding activation maps, the method generates a faithful description of the classifier's decision process in the form of a structured meaning representation, which is then converted into text by a language model. Through this pipeline approach, the generated explanations are grounded in the neural network architecture, providing accurate insight into the classification process while remaining accessible to non-experts. Experimental results show that the NLEs constructed by our method are significantly more plausible and faithful. In particular, user interventions in the neural network structure (masking of neurons) are three times more effective than the baselines.