# Verbosity Tradeoffs and the Impact of Scale on the Faithfulness of LLM Self-Explanations

arXiv: https://arxiv.org/abs/2503.13445 · HTML: https://mlsanity.com/paper/2503.13445/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2025.

## Built on by (most cited first, 7 of 7)

- [A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behavior](https://mlsanity.com/paper/2602.02639/) (2026)
- [Biases in the Blind Spot: Detecting What LLMs Fail to Mention](https://mlsanity.com/paper/2602.10117/) (2026)
- [Can Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanations](https://mlsanity.com/paper/2601.00282/) (2026)
- [Evaluating and Improving LLM Self-Modeling](https://mlsanity.com/paper/2608.30980/) (2026)
- [Faithful by Definition: Emotion Analysis via Natural Semantic Metalanguage Explications](https://mlsanity.com/paper/2607.00661/) (2026)
- [Counterfactual Tests for Measuring Chain-of-Thought Faithfulness in Visual Language Models](https://mlsanity.com/paper/2609.06704/) (2026)
- [An Empirical Study of Counterfactual Self-Explanations in LLMs](https://mlsanity.com/paper/2609.17119/) (2026)

## Abstract (arXiv)

When asked to explain their decisions, LLMs can often give explanations which sound plausible to humans. But are these explanations faithful, i.e. do they convey the factors actually responsible for the decision? In this work, we analyse counterfactual faithfulness across 75 models from 13 families. We analyze the tradeoff between conciseness and comprehensiveness, how correlational faithfulness metrics assess this tradeoff, and the extent to which metrics can be gamed. This analysis motivates two new metrics: the phi-CCT, a simplified variant of the Correlational Counterfactual Test (CCT) which avoids the need for token probabilities while explaining most of the variance of the original test; and F-AUROC, which eliminates sensitivity to imbalanced intervention distributions and captures a model's ability to produce explanations with different levels of detail. Our findings reveal a clear scaling trend: larger and more capable models are consistently more faithful on all metrics we consider. Our code is available at https://github.com/google-deepmind/corr_faith.