mlsanityData to 7 Oct 2026

Interpretation of NLP models through input marginalization

arXiv 2010.13984 ↗First posted 27 Oct 2020cs.CL cs.AIcited by 43

Not on the map. Why?

Abstract (arXiv)

To demystify the "black box" property of deep neural networks for natural language processing (NLP), several methods have been proposed to interpret their predictions by measuring the change in prediction probability after erasing each token of an input. Since existing methods replace each token with a predefined value (i.e., zero), the resulting sentence lies out of the training data distribution, yielding misleading interpretations. In this study, we raise the out-of-distribution problem induced by the existing interpretation methods and present a remedy; we propose to marginalize each token out. We interpret various NLP models trained for sentiment analysis and natural language inference using the proposed method.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 10; 2023: 8; 2024: 9; 2025: 1; 2026: 3.

43 papers.

Sort:

By area:

  1. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 110
  2. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 86
  3. 2021Contrastive Explanations for Model Interpretabilitycited by 75
  4. 2021"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classificationcited by 66
  5. 2023ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writingcited by 31
  6. 2022Grounding Visual Representations with Texts for Domain Generalizationcited by 30
  7. 2024Optimal ablation for interpretabilitycited by 21
  8. 2022Learning to Scaffold: Optimizing Model Explanations for Teachingcited by 20
  9. 2023Generative Calibration for In-context Learningcited by 20
  10. 2021Interpreting Deep Learning Models in Natural Language Processing: A Reviewcited by 19
  11. 2021Jointly Attacking Graph Neural Network and its Explanationscited by 18
  12. 2021Logic Traps in Evaluating Attribution Scorescited by 18
31 more
  1. 2022Necessity and Sufficiency for Explaining Text Classifiers: A Case Study in Hate Speech Detectioncited by 18
  2. 2021PredDiff: Explanations and Interactions from Conditional Expectationscited by 16
  3. 2022Towards Explainable Evaluation Metrics for Natural Language Generationcited by 15
  4. 2021On Sample Based Explanation Methods for NLP:Efficiency, Faithfulness, and Semantic Evaluationcited by 12
  5. 2022The Solvability of Interpretability Evaluation Metricscited by 11
  6. 2024Multi-Level Explanations for Generative Language Modelscited by 11
  7. 2024Hard to Explain: On the Computational Hardness of In-Distribution Model Interpretationcited by 11
  8. 2021Flexible Instance-Specific Rationalization of NLP Modelscited by 10
  9. 2022Deconfounding to Explanation Evaluation in Graph Neural Networkscited by 10
  10. 2024Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitationscited by 10
  11. 2022How explainable are adversarially-robust CNNs?cited by 9
  12. 2024Facilitating Human-LLM Collaboration through Factuality Scores and Source Attributionscited by 9
  13. 2023Feature Interactions Reveal Linguistic Structure in Language Modelscited by 8
  14. 2023Explaining with Contrastive Phrasal Highlighting: A Case Study in Assisting Humans to Detect Translation Differencescited by 8
  15. 2021Double Trouble: How to not explain a text classifier's decisions using counterfactuals synthesized by masked language models?cited by 5
  16. 2022Hierarchical Interpretation of Neural Text Classificationcited by 5
  17. 2022A Multilingual Perspective Towards the Evaluation of Attribution Methods in Natural Language Inferencecited by 5
  18. 2023A Multi-Grained Self-Interpretable Symbolic-Neural Model For Single/Multi-Labeled Text Classificationcited by 4
  19. 2023CUE: An Uncertainty Interpretation Framework for Text Classifiers Built on Pre-Trained Language Modelscited by 3
  20. 2024Counterfactuals As a Means for Evaluating Faithfulness of Attribution Methods in Autoregressive Language Modelscited by 3
  21. 2026An Efficient Streaming Video Understanding Framework with Agentic Controlcited by 3
  22. 2021Overcoming Poor Word Embeddings with Word Definitionscited by 1
  23. 2021Does BERT look at sentiment lexicon?cited by 1
  24. 2023Are Good Explainers Secretly Human-in-the-Loop Active Learners?cited by 1
  25. 2024InfFeed: Influence Functions as a Feedback to Improve the Performance of Subjective Taskscited by 1
  26. 2026Attention-guided Fine-tuning of Multimodal Large Language Models Improves Chain-of-Thought Reasoningcited by 1
  27. 2022BMX: Boosting Natural Language Generation Metrics with Explainabilitycited by 0
  28. 2023Interpreting Sentiment Composition with Latent Semantic Treecited by 0
  29. 2024Using Interpretation Methods for Model Enhancementcited by 0
  30. 2025Splits! Flexible Sociocultural Linguistic Investigation at Scalecited by 0
  31. 2026Missingness Bias Calibration in Feature Attribution Explanationscited by 0