# An Empirical Study of Counterfactual Self-Explanations in LLMs

arXiv: https://arxiv.org/abs/2609.17119 · HTML: https://mlsanity.com/paper/2609.17119/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 15 Sep 2026.

## Builds on (most cited first, 12 of 12)

- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [Faithfulness Tests for Natural Language Explanations](https://mlsanity.com/paper/2305.18029/) (2023)
- [Are self-explanations from Large Language Models faithful?](https://mlsanity.com/paper/2401.07927/) (2024)
- [FaithLM: Towards Faithful Explanations for Large Language Models](https://mlsanity.com/paper/2402.04678/) (2024)
- [Zero-shot LLM-guided Counterfactual Generation: A Case Study on NLP Model Evaluation](https://mlsanity.com/paper/2405.04793/) (2024)
- [LLMs Don't Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanations](https://mlsanity.com/paper/2509.09396/) (2025)
- [Verbosity Tradeoffs and the Impact of Scale on the Faithfulness of LLM Self-Explanations](https://mlsanity.com/paper/2503.13445/) (2025)
- [Do LLM Self-Explanations Help Users Predict Model Behavior? Evaluating Counterfactual Simulatability with Pragmatic Perturbations](https://mlsanity.com/paper/2601.03775/) (2026)
- [Counterfactuals of Counterfactuals: a back-translation-inspired approach to analyse counterfactual editors](https://mlsanity.com/paper/2305.17055/) (2023)

## Abstract (arXiv)

Large language models can easily generate explanations for their own outputs, but such self-explanations are not necessarily faithful to the model's behavior. We study this issue through counterfactual self-explanations, where a model minimally edits an input so that its own prediction changes. Across sentiment analysis and natural language inference, we evaluate ten instruction-tuned models from the LLaMA-3 and Qwen-2.5 families, measuring faithfulness, minimality, and alignment with human-annotated rationales. Our results show that model scale is the strongest determinant of explanation quality: larger models are substantially more likely to generate counterfactuals that flip their own predictions and target decision-relevant evidence. In contrast, the rationale-guided condition produces edit-minimal counterfactuals that are also more human-aligned. However, it does not consistently improve faithfulness. Overall, counterfactual self-explanations can provide useful behavioral evidence about model decisions, but their reliability depends strongly on model capacity and should be empirically validated rather than assumed.