mlsanityData to 7 Oct 2026

Local Interpretations for Explainable Natural Language Processing: A Survey

arXiv 2103.11072 ↗First posted 20 Mar 2021cs.CL cs.AIcited by 31

Not on the map. Why?

Abstract (arXiv)

As the use of deep learning techniques has grown across various fields over the past decade, complaints about the opaqueness of the black-box models have increased, resulting in an increased focus on transparency in deep learning models. This work investigates various methods to improve the interpretability of deep neural networks for Natural Language Processing (NLP) tasks, including machine translation and sentiment analysis. We provide a comprehensive discussion on the definition of the term interpretability and its various aspects at the beginning of this work. The methods collected and summarised in this survey are only associated with local interpretation and are specifically divided into three categories: 1) interpreting the model's predictions through related input features; 2) interpreting through natural language explanation; 3) probing the hidden states of models and word representations.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 126 of its 206 references that are in the map. First posted on arXiv 20 Mar 2021.

Builds on

126 of 206 references.

References · 126

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  7. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  8. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  9. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  10. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  11. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  12. 2017Axiomatic Attribution for Deep Networkscited by 3,350
38 more
  1. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  2. 2015VQA: Visual Question Answeringcited by 2,489
  3. 2018Deep contextualized word representationscited by 2,285
  4. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  5. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  6. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  7. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  8. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  9. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  10. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  11. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  12. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  13. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  14. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  15. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  16. 2019Attention is not Explanationcited by 916
  17. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  18. 2016The Mythos of Model Interpretabilitycited by 716
  19. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  20. 2019Attention is not not Explanationcited by 629
  21. 2019Designing and Interpreting Probes with Control Taskscited by 608
  22. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  23. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 571
  24. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  25. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  26. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  27. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  28. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 440
  29. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  30. 2019Is Attention Interpretable?cited by 387
  31. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 382
  32. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  33. 2019Deep Modular Co-Attention Networks for Visual Question Answeringcited by 338
  34. 2019Interpretable machine learning: definitions, methods, and applicationscited by 303
  35. 2016Rationalizing Neural Predictionscited by 281
  36. 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278
  37. 2018Analysis Methods in Neural Language Processing: A Surveycited by 276
  38. 2015Stacked Attention Networks for Image Question Answeringcited by 273

50 most cited of 126 All 126 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 5; 2024: 9; 2025: 5; 2026: 4.

31 papers.

Sort:

By area:

  1. 2023Explainability for Large Language Models: A Surveycited by 331
  2. 2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationalescited by 102
  3. 2021UNIREX: A Unified Learning Framework for Language Model Rationale Extractioncited by 32
  4. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 24
  5. 2021Ethics Sheet for Automatic Emotion Recognition and Sentiment Analysiscited by 22
  6. 2023Rationalization for Explainable NLP: A Surveycited by 22
  7. 2023Multimodal Data Integration for Oncology in the Era of Deep Neural Networks: A Reviewcited by 20
  8. 2022Necessity and Sufficiency for Explaining Text Classifiers: A Case Study in Hate Speech Detectioncited by 18
  9. 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
  10. 2022Doc-GCN: Heterogeneous Graph Convolutional Networks for Document Layout Analysiscited by 17
  11. 2024F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AIcited by 17
  12. 2022Exploring Faithful Rationale for Multi-hop Fact Verification via Salience-Aware Graph Learningcited by 15
19 more
  1. 2022ER-Test: Evaluating Explanation Regularization Methods for Language Modelscited by 6
  2. 2024Penzai + Treescope: A Toolkit for Interpreting, Visualizing, and Editing Models As Datacited by 5
  3. 2025Explainable Sentiment Analysis with DeepSeek-R1: Performance, Efficiency, and Few-Shot Learningcited by 5
  4. 2022Understanding Attention for Vision-and-Language Taskscited by 4
  5. 2023ExClaim: Explainable Neural Claim Verification Using Rationalizationcited by 4
  6. 2023REFER: An End-to-end Rationale Extraction Framework for Explanation Regularizationcited by 4
  7. 2024SIDU-TXT: An XAI Algorithm for NLP with a Holistic Assessment Approachcited by 4
  8. 2024Tell Me Why: Explainable Public Health Fact-Checking with Large Language Modelscited by 3
  9. 2025From Fragment to One Piece: A Survey on AI-Driven Graphic Designcited by 2
  10. 2025Multi-Domain Explainability of Preferencescited by 2
  11. 2025EMMM, Explain Me My Model! Explainable Machine Generated Text Detection in Dialoguescited by 2
  12. 2024DeepCodeProbe: Towards Understanding What Models Trained on Code Learncited by 1
  13. 2024Make Interval Bound Propagation great againcited by 1
  14. 2024FarExStance: Explainable Stance Detection for Farsicited by 1
  15. 2026LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactualscited by 1
  16. 2022Reconnoitering the class distinguishing abilities of the features, to know them bettercited by 0
  17. 2026The Dynamic Gist-Based Memory Model (DGMM): A Memory-Centric Architecture for Artificial Intelligencecited by 0
  18. 2026MIL-BERT: Classification of Arbitrarily Large Text with Performance and Explanatory Guaranteescited by 0
  19. 2026Why Current XAI Is Not Enough for Arabic NLP: A Critical Survey of the Explainability Gapcited by 0