mlsanityData to 7 Oct 2026

On the Sensitivity and Stability of Model Interpretations in NLP

arXiv 2104.08782 ↗First posted 18 Apr 2021cs.CLcited by 30

Not on the map. Why?

Abstract (arXiv)

Recent years have witnessed the emergence of a variety of post-hoc interpretations that aim to uncover how natural language processing (NLP) models make predictions. Despite the surge of new interpretation methods, it remains an open problem how to define and quantitatively measure the faithfulness of interpretations, i.e., to what extent interpretations reflect the reasoning process by a model. We propose two new criteria, sensitivity and stability, that provide complementary notions of faithfulness to the existed removal-based criteria. Our results show that the conclusion for how faithful interpretations are could vary substantially based on different notions. Motivated by the desiderata of sensitivity and stability, we introduce a new class of interpretation methods that adopt techniques from adversarial robustness. Empirical results show that our proposed methods are effective under the new criteria and overcome limitations of gradient-based methods on removal-based criteria. Besides text classification, we also apply interpretation methods and metrics to dependency parsing. Our results shed light on understanding the diverse set of interpretations.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 35 of its 47 references that are in the map. First posted on arXiv 18 Apr 2021.

Builds on

35 of 47 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  8. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  9. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  10. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  11. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  12. 2019Attention is not Explanationcited by 916
23 more
  1. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  2. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  3. 2017Interpretation of Neural Networks is Fragilecited by 443
  4. 2018Generating Natural Language Adversarial Examplescited by 422
  5. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  6. 2019Is Attention Interpretable?cited by 387
  7. 2016Deep Biaffine Attention for Neural Dependency Parsingcited by 334
  8. 2017The (Un)reliability of saliency methodscited by 331
  9. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 322
  10. 2018On the Robustness of Interpretability Methodscited by 310
  11. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  12. 2015Visualizing and Understanding Neural Models in NLPcited by 199
  13. 2020Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?cited by 189
  14. 2019Certified Robustness to Adversarial Word Substitutionscited by 164
  15. 2020A Diagnostic Study of Explainability Techniques for Text Classificationcited by 158
  16. 2018Pathologies of Neural Models Make Interpretations Difficultcited by 140
  17. 2020The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?cited by 122
  18. 2020Evaluating Explanations: How much do explanations from the teacher aid students?cited by 94
  19. 2019Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagationcited by 87
  20. 2020Robustness Verification for Transformerscited by 76
  21. 2020Evaluations and Methods for Explanation through Robustness Analysiscited by 51
  22. 2021Evaluating Saliency Methods for Neural Language Modelscited by 37
  23. 2020Are Interpretations Fairly Evaluated? A Definition Driven Pipeline for Post-Hoc Interpretabilitycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 7; 2023: 8; 2024: 9; 2025: 4; 2026: 1.

30 papers.

Sort:

By area:

  1. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  2. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  3. 2021"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classificationcited by 66
  4. 2023Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactualscited by 44
  5. 2022Don't Lie to Me! Robust and Efficient Explainability with Verified Perturbation Analysiscited by 38
  6. 2022ferret: a Framework for Benchmarking Explainers on Transformerscited by 26
  7. 2022SEAT: Stable and Explainable Attentioncited by 25
  8. 2024SATO: Stable Text-to-Motion Frameworkcited by 20
  9. 2022Generic Temporal Reasoning with Differential Analysis and Explanationcited by 18
  10. 2024F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AIcited by 17
  11. 2022A Fine-grained Interpretability Evaluation Benchmark for Neural NLPcited by 15
  12. 2023Incorporating Attribution Importance for Improving Faithfulness Metricscited by 14
18 more
  1. 2024Prompt-Agnostic Adversarial Perturbation for Customized Diffusion Modelscited by 14
  2. 2023Improving Interpretation Faithfulness for Vision Transformerscited by 12
  3. 2024Evaluating Human Alignment and Model Faithfulness of LLM Rationalecited by 12
  4. 2023ALMANACS: A Simulatability Benchmark for Language Model Explainabilitycited by 11
  5. 2024Normalized AOPC: Fixing Misleading Faithfulness Metrics for Feature Attribution Explainabilitycited by 10
  6. 2023On the Impact of Knowledge Distillation for Model Interpretabilitycited by 8
  7. 2023Human Attention-Guided Explainable Artificial Intelligence for Computer Vision Modelscited by 6
  8. 2024DRIVE: Dependable Robust Interpretable Visionary Ensemble Framework in Autonomous Drivingcited by 5
  9. 2022An Interpretability Evaluation Benchmark for Pre-trained Language Modelscited by 4
  10. 2023Efficient Shapley Values Estimation by Amortization for Text Classificationcited by 4
  11. 2024Faithful Interpretation for Graph Neural Networkscited by 3
  12. 2024CERET: Cost-Effective Extrinsic Refinement for Text Generationcited by 1
  13. 2025Explainable AI the Latest Advancements and New Trendscited by 1
  14. 2024Influence based explainability of brain tumors segmentation in multimodal Magnetic Resonance Imagingcited by 0
  15. 2025When Machine Learning Gets Personal: Evaluating Prediction and Explanationcited by 0
  16. 2025Multilingual Relative Clause Attachment Ambiguity Resolution in Large Language Modelscited by 0
  17. 2025Application of integrated gradients explainability to sociopsychological semantic markerscited by 0
  18. 2026Faithfulness Evaluation for Decoder-only LLM Attributions with Controlled Retained Informationcited by 0