mlsanityData to 7 Oct 2026

Are self-explanations from Large Language Models faithful?

arXiv 2401.07927 ↗First posted 15 Jan 2024cs.CL cs.AI cs.LGcited by 101

Not on the map. Why?

Abstract (arXiv)

Instruction-tuned Large Language Models (LLMs) excel at many tasks and will even explain their reasoning, so-called self-explanations. However, convincing and wrong self-explanations can lead to unsupported confidence in LLMs, thus increasing risk. Therefore, it's important to measure if self-explanations truly reflect the model's behavior. Such a measure is called interpretability-faithfulness and is challenging to perform since the ground truth is inaccessible, and many LLMs only have an inference API. To address this, we propose employing self-consistency checks to measure faithfulness. For example, if an LLM says a set of words is important for making a prediction, then it should not be able to make its prediction without these words. While self-consistency checks are a common approach to faithfulness, they have not previously been successfully applied to LLM self-explanations for counterfactual, feature attribution, and redaction explanations. Our results demonstrate that faithfulness is explanation, model, and task-dependent, showing self-explanations should not be trusted in general. For example, with sentiment classification, counterfactuals are more faithful for Llama2, feature attribution for Mistral, and redaction for Falcon 40B.

Key facts. Cited by 101 papers in the map (data through 7 Oct 2026). Builds on 45 of its 58 references that are in the map. First posted on arXiv 15 Jan 2024.

Builds on

45 of 58 references.

References · 45

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  5. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  6. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  7. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  8. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  9. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  10. 2022A Survey on In-context Learningcited by 717
  11. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  12. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
33 more
  1. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  2. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  3. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
  4. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  5. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  6. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  7. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  8. 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 363
  9. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  10. 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 309
  11. 2016Rationalizing Neural Predictionscited by 281
  12. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 219
  13. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  14. 2015Visualizing and Understanding Recurrent Networkscited by 158
  15. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  16. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  17. 2023Hallucinations in Large Multilingual Translation Modelscited by 126
  18. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  19. 2020Explaining NLP Models via Minimal Contrastive Editing (MiCE)cited by 105
  20. 2023SCOTT: Self-Consistent Chain-of-Thought Distillationcited by 103
  21. 2020NILE : Natural Language Inference with Faithful Natural Language Explanationscited by 96
  22. 2020Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?cited by 95
  23. 2023Can Large Language Models Explain Themselves? A Study of LLM-Generated Self-Explanationscited by 90
  24. 2023Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanationscited by 85
  25. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  26. 2023Copy Suppression: Comprehensively Understanding an Attention Headcited by 73
  27. 2021"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classificationcited by 66
  28. 2021Towards Few-Shot Fact-Checking via Perplexitycited by 63
  29. 2018Towards Explainable NLP: A Generative Explanation Framework for Text Classificationcited by 55
  30. 2023Benchmarking and Improving Generator-Validator Consistency of Language Modelscited by 43
  31. 2020Explaining Question Answering Models through Text Generationcited by 33
  32. 2021Evaluating the Faithfulness of Importance Measures in NLP by Recursively Masking Allegedly Important Tokens and Retrainingcited by 31
  33. 2022Using Interactive Feedback to Improve the Accuracy and Explainability of Question Answering Systems Post-Deploymentcited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 21; 2025: 35; 2026: 45.

101 papers.

Sort:

By area:

  1. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  2. 2025Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Abovecited by 51
  3. 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Stepscited by 43
  4. 2026This human study did not involve human subjects: Validating LLM simulations as behavioral evidencecited by 26
  5. 2024Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generationcited by 25
  6. 2024FaithLM: Towards Faithful Explanations for Large Language Modelscited by 18
  7. 2024Interpretable LLM-based Table Question Answeringcited by 18
  8. 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
  9. 2025LLMs Don't Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanationscited by 16
  10. 2025Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missingcited by 14
  11. 2025Training a Utility-based Retriever Through Shared Context Attribution for Retrieval-Augmented Language Modelscited by 13
  12. 2024Evaluating Human Alignment and Model Faithfulness of LLM Rationalecited by 12
38 more
  1. 2024Multi-Level Explanations for Generative Language Modelscited by 11
  2. 2025Learning to Attribute with Attentioncited by 11
  3. 2025Robust Answers, Fragile Logic: Probing the Decoupling Hypothesis in LLM Reasoningcited by 11
  4. 2024Interpretability Needs a New Paradigmcited by 10
  5. 2024Unearthing Skill-Level Insights for Understanding Trade-Offs of Foundation Modelscited by 9
  6. 2025Truthful or Fabricated? Using Causal Attribution to Mitigate Reward Hacking in Explanationscited by 9
  7. 2025LExT: Towards Evaluating Trustworthiness of Natural Language Explanationscited by 9
  8. 2025Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Frameworkcited by 9
  9. 2024Self-AMPLIFY: Improving Small Language Models with Self Post Hoc Explanationscited by 8
  10. 2024Don't be Fooled: The Misinformation Effect of Explanations in Human-AI Collaborationcited by 8
  11. 2024Semi-supervised Fine-tuning for Large Language Modelscited by 8
  12. 2026Loop as a Bridge: Can Looped Transformers Truly Link Representation Space and Natural Language Outputs?cited by 8
  13. 2025Counterfactual Simulatability of LLM Explanations for Generation Taskscited by 7
  14. 2026Do LLM Self-Explanations Help Users Predict Model Behavior? Evaluating Counterfactual Simulatability with Pragmatic Perturbationscited by 7
  15. 2025Translate Policy to Language: Flow Matching Generated Rewards for LLM Explanationscited by 6
  16. 2025Concept-Level Explainability for Auditing & Steering LLM Responsescited by 6
  17. 2025Towards Transparent Reasoning: What Drives Faithfulness in Large Language Models?cited by 5
  18. 2026Can Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanationscited by 5
  19. 2024Do Robot Snakes Dream like Electric Sheep? Investigating the Effects of Architectural Inductive Biases on Hallucinationcited by 4
  20. 2025B-cos LM: Efficiently Transforming Pre-trained Language Models for Improved Explainabilitycited by 4
  21. 2025Aligning What LLMs Do and Say: Towards Self-Consistent Explanationscited by 4
  22. 2026LLMs Can't Play Hangman: On the Necessity of a Private Working Memory for Language Agentscited by 4
  23. 2026Verbalizing LLM's Higher-order Uncertainty via Imprecise Probabilitiescited by 4
  24. 2026Most LLM Conformity Needs No Speaker: Measuring the Speaker-Free Floor in Peer-Pressure Benchmarkscited by 4
  25. 2024Low-rank finetuning for LLMs: A fairness perspectivecited by 3
  26. 2024Linearly-Interpretable Concept Embedding Models for Text Analysiscited by 3
  27. 2024Evaluating the Reliability of Self-Explanations in Large Language Modelscited by 3
  28. 2024Using Instruction-Tuned Large Language Models to Identify Indicators of Vulnerability in Police Incident Narrativescited by 3
  29. 2025Learning to Explain: Prototype-Based Surrogate Models for LLM Classificationcited by 3
  30. 2025PEDANTIC: A Dataset for the Automatic Examination of Definiteness in Patent Claimscited by 3
  31. 2025On the Performance of LLMs for Real Estate Appraisalcited by 3
  32. 2025Interpreting Language Models Through Concept Descriptions: A Surveycited by 3
  33. 2026Overseeing Agents Without Constant Oversight: Challenges and Opportunitiescited by 3
  34. 2026Training Agents to Self-Report Misbehaviorcited by 3
  35. 2026Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervisioncited by 3
  36. 2025Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoringcited by 2
  37. 2025Is Long-to-Short a Free Lunch? Investigating Inconsistency and Reasoning Efficiency in LRMscited by 2
  38. 2025Interpreting LLM-as-a-Judge Policies via Verifiable Global Explanationscited by 2

50 most cited of 101 All 101 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023On Measuring Faithfulness or Self-consistency of Natural Language Explanations
  2. 2024Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models
  3. 2024The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models
  4. 2025A Causal Lens for Evaluating Faithfulness Metrics
  5. 2026A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behavior
  6. 2026Training Large Language Models for Self-Explanation Faithfulness
  7. 2025Self-Critique and Refinement for Faithful Natural Language Explanations
  8. 2026When Should a Language Model Trust Itself? Same-Model Self-Verification as a Conditional Confidence Signal