mlsanityData to 7 Oct 2026

Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?

arXiv 2004.03685 ↗First posted 7 Apr 2020cs.CL cs.LGcited by 590

Not on the map. Why?

Cited by 68 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

With the growing popularity of deep-learning based NLP models, comes a need for interpretable systems. But what is interpretability, and what constitutes a high-quality interpretation? In this opinion piece we reflect on the current state of interpretability evaluation research. We call for more clearly differentiating between different desired criteria an interpretation should satisfy, and focus on the faithfulness criteria. We survey the literature with respect to faithfulness evaluation, and arrange the current approaches around three assumptions, providing an explicit form to how faithfulness is "defined" by the community. We provide concrete guidelines on how evaluation of interpretation methods should and should not be conducted. Finally, we claim that the current binary definition for faithfulness sets a potentially unrealistic bar for being considered faithful. We call for discarding the binary notion of faithfulness in favor of a more graded one, which we believe will be of greater practical utility.

Key facts. Cited by 590 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 57; 2022: 89; 2023: 79; 2024: 89; 2025: 120; 2026: 156.

590 papers.

Sort:

By area:

  1. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  2. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  3. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  4. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  5. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  6. 2025Reasoning Models Don't Always Say What They Thinkcited by 388
  7. 2021A Survey on Automated Fact-Checkingcited by 376
  8. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 338
  9. 2023Explainability for Large Language Models: A Surveycited by 331
  10. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  11. 2022Teaching language models to support answers with verified quotescited by 290
  12. 2023Faithful Chain-of-Thought Reasoningcited by 285
38 more
  1. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
  2. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 197
  3. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  4. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 177
  5. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithfulcited by 166
  6. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  7. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  8. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  9. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  10. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
  11. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
  12. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  13. 2021Truthful AI: Developing and governing AI that does not liecited by 109
  14. 2023A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimationcited by 108
  15. 2024Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoningcited by 108
  16. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  17. 2023Question Decomposition Improves the Faithfulness of Model-Generated Reasoningcited by 107
  18. 2024Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Modelscited by 104
  19. 2024Are self-explanations from Large Language Models faithful?cited by 101
  20. 2023Tracr: Compiled Transformers as a Laboratory for Interpretabilitycited by 99
  21. 2021Do Feature Attribution Methods Correctly Attribute Features?cited by 98
  22. 2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
  23. 2022Evaluating Human-Language Model Interactioncited by 90
  24. 2023Answering Questions by Meta-Reasoning over Multiple Chains of Thoughtcited by 87
  25. 2023Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanationscited by 85
  26. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  27. 2021Explanations in Autonomous Driving: A Surveycited by 80
  28. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  29. 2021Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Casescited by 77
  30. 2023Boosting Language Models Reasoning with Chain-of-Knowledge Promptingcited by 77
  31. 2021Contrastive Explanations for Model Interpretabilitycited by 75
  32. 2023Copy Suppression: Comprehensively Understanding an Attention Headcited by 73
  33. 2022GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspectivecited by 72
  34. 2023A Survey of Large Language Models Attributioncited by 71
  35. 2023Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaborationcited by 70
  36. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 67
  37. 2021"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classificationcited by 66
  38. 2023HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attributioncited by 55

50 most cited of 590 All 590 (JSON)