mlsanityData to 7 Oct 2026

Towards Faithful Model Explanation in NLP: A Survey

arXiv 2209.11326 ↗First posted 22 Sep 2022cs.CLcited by 129

Not on the map. Why?

Abstract (arXiv)

End-to-end neural Natural Language Processing (NLP) models are notoriously difficult to understand. This has given rise to numerous efforts towards model explainability in recent years. One desideratum of model explanation is faithfulness, i.e. an explanation should accurately represent the reasoning process behind the model's prediction. In this survey, we review over 110 model explanation methods in NLP through the lens of faithfulness. We first discuss the definition and evaluation of faithfulness, as well as its significance for explainability. We then introduce recent advances in faithful explanation, grouping existing approaches into five categories: similarity-based methods, analysis of model-internal structures, backpropagation-based methods, counterfactual intervention, and self-explanatory models. For each category, we synthesize its representative studies, strengths, and weaknesses. Finally, we summarize their common virtues and remaining challenges, and reflect on future work directions towards faithful explainability in NLP.

Key facts. Cited by 129 papers in the map (data through 7 Oct 2026). Builds on 219 of its 272 references that are in the map. First posted on arXiv 22 Sep 2022.

Builds on

219 of 272 references.

References · 219

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  10. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  11. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  12. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
38 more
  1. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  2. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  3. 2015VQA: Visual Question Answeringcited by 2,489
  4. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  5. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  6. 2019Language Models as Knowledge Bases?cited by 1,922
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  8. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  9. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  10. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  11. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  12. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  13. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  14. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  15. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  16. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  17. 2018Sanity Checks for Saliency Mapscited by 1,149
  18. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  19. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  20. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  21. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  22. 2019Attention is not Explanationcited by 916
  23. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  24. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  25. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  26. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  27. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  28. 2020Quantifying Attention Flow in Transformerscited by 759
  29. 2016The Mythos of Model Interpretabilitycited by 716
  30. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  31. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  32. 2019Attention is not not Explanationcited by 629
  33. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  34. 2019Designing and Interpreting Probes with Control Taskscited by 608
  35. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  36. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 542
  37. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  38. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505

50 most cited of 219 All 219 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 22; 2024: 37; 2025: 43; 2026: 27.

129 papers.

Sort:

By area:

  1. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  2. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 197
  3. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  4. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
  5. 2023Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanationscited by 85
  6. 2024LLMs Will Always Hallucinate, and We Need to Live With Thiscited by 77
  7. 2023On Measuring Faithfulness or Self-consistency of Natural Language Explanationscited by 71
  8. 2024A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Modelscited by 52
  9. 2023Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactualscited by 44
  10. 2023DecompX: Explaining Transformers Decisions by Propagating Token Decompositioncited by 31
  11. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  12. 2024Codebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Conceptscited by 28
38 more
  1. 2024Chain-of-Thought Unfaithfulness as Disguised Accuracycited by 27
  2. 2024Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generationcited by 25
  3. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 24
  4. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
  5. 2023DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformerscited by 22
  6. 2024Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agendacited by 22
  7. 2023Interpretable-by-Design Text Understanding with Iteratively Generated Concept Bottleneckcited by 21
  8. 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
  9. 2024F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AIcited by 17
  10. 2023Atomic Inference for NLI with Generated Facts as Atomscited by 16
  11. 2023Language Models with Rationalitycited by 15
  12. 2023Interpretation of Time-Series Deep Models: A Surveycited by 15
  13. 2025MetaFaith: Faithful Natural Language Uncertainty Expression in LLMscited by 15
  14. 2023Incorporating Attribution Importance for Improving Faithfulness Metricscited by 14
  15. 2023Stability Guarantees for Feature Attributions with Multiplicative Smoothingcited by 14
  16. 2024Latent Concept-based Explanation of NLP Modelscited by 13
  17. 2023Improving Interpretation Faithfulness for Vision Transformerscited by 12
  18. 2024An Unsupervised Approach to Achieve Supervised-Level Explainability in Healthcare Recordscited by 12
  19. 2025A Causal Lens for Evaluating Faithfulness Metricscited by 12
  20. 2025Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variationscited by 12
  21. 2025ViPER: Empowering the Self-Evolution of Visual Perception Abilities in Vision-Language Modelcited by 11
  22. 2024Interpretability Needs a New Paradigmcited by 10
  23. 2024Normalized AOPC: Fixing Misleading Faithfulness Metrics for Feature Attribution Explainabilitycited by 10
  24. 2024Intrinsic User-Centric Interpretability through Global Mixture of Expertscited by 9
  25. 2025LExT: Towards Evaluating Trustworthiness of Natural Language Explanationscited by 9
  26. 2025Probabilistic Stability Guarantees for Feature Attributionscited by 9
  27. 2025textttDroid: A Resource Suite for AI-Generated Code Detectioncited by 9
  28. 2026RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Modelscited by 9
  29. 2024Beyond Agreement: Diagnosing the Rationale Alignment of Automated Essay Scoring Methods based on Linguistically-informed Counterfactualscited by 8
  30. 2024Cross-Refine: Improving Natural Language Explanation Generation by Learning in Tandemcited by 8
  31. 2023Sum-of-Parts: Self-Attributing Neural Networks with End-to-End Learning of Feature Groupscited by 6
  32. 2024Navigating the Landscape of Hint Generation Research: From the Past to the Futurecited by 6
  33. 2024Interactive Prompt Debugging with Sequence Saliencecited by 6
  34. 2024The Illusion of Competence: Evaluating the Effect of Explanations on Users' Mental Models of Visual Question Answering Systemscited by 6
  35. 2025Towards Achieving Concept Completeness for Textual Concept Bottleneck Modelscited by 6
  36. 2023Multilingual Nonce Dependency Treebanks: Understanding how Language Models represent and process syntactic structurecited by 5
  37. 2024DRIVE: Dependable Robust Interpretable Visionary Ensemble Framework in Autonomous Drivingcited by 5
  38. 2024LibraGrad: Balancing Gradient Flow for Universally Better Vision Transformer Attributionscited by 5

50 most cited of 129 All 129 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models
  2. 2025Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations
  3. 2023Benchmarking Faithfulness: Towards Accurate Natural Language Explanations in Vision-Language Tasks
  4. 2025Self-Critique and Refinement for Faithful Natural Language Explanations
  5. 2024The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models
  6. 2025Reasoning-Grounded Natural Language Explanations for Language Models
  7. 2026ICE: Intervention-Consistent Explanation Evaluation with Statistical Grounding for LLMs
  8. 2024Are self-explanations from Large Language Models faithful?