mlsanityData to 14 Oct 2026

Interpreting Language Models with Contrastive Explanations

arXiv 2202.10419 ↗First posted 21 Feb 2022cs.CL cs.AI cs.LGcited by 92

Not on the map. Why?

Abstract (arXiv)

Model interpretability methods are often used to explain NLP model decisions on tasks such as text classification, where the output space is relatively small. However, when applied to language generation, where the output space often consists of tens of thousands of tokens, these methods are unable to provide informative explanations. Language models must consider various features to predict a token, such as its part of speech, number, tense, or semantics. Existing explanation methods conflate evidence for all these features into a single explanation, which is less interpretable for human understanding. To disentangle the different decisions in language modeling, we focus on explaining language models contrastively: we look for salient input tokens that explain why the model predicted one token instead of another. We demonstrate that contrastive explanations are quantifiably better than non-contrastive explanations in verifying major grammatical phenomena, and that they significantly improve contrastive model simulatability for human observers. We also identify groups of contrastive decisions where the model uses similar evidence, and we are able to characterize what input tokens models use during various language generation decisions.

Key facts. Cited by 92 papers in the map (data through 14 Oct 2026). Builds on 23 of its 36 references that are in the map. First posted on arXiv 21 Feb 2022.

Builds on

23 of 36 references.

References · 23

  1. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  2. 2016Pointer Sentinel Mixture Modelscited by 3,096
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  4. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  5. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  6. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
  7. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 297
  8. 2018Analysis Methods in Neural Language Processing: A Surveycited by 277
  9. 2016Understanding Neural Networks through Representation Erasurecited by 224
  10. 2018Marian: Fast Neural Machine Translation in C++cited by 203
  11. 2015Visualizing and Understanding Neural Models in NLPcited by 200
  12. 2020Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?cited by 190
11 more
  1. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 156
  2. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 131
  3. 2020Evaluating Explanations: How much do explanations from the teacher aid students?cited by 94
  4. 2021Contrastive Explanations for Model Interpretabilitycited by 75
  5. 2019AllenNLP Interpret: A Framework for Explaining Predictions of NLP Modelscited by 74
  6. 2021Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Predictioncited by 59
  7. 2018Do Explanations make VQA Models more Predictable to a Human?cited by 52
  8. 2014Extraction of Salient Sentences from Labelled Documentscited by 50
  9. 2021When Does Translation Require Context? A Data-driven, Multilingual Explorationcited by 36
  10. 2019Fine-grained Sentiment Analysis with Faithful Attentioncited by 23
  11. 2021Do Context-Aware Translation Models Pay the Right Attention?cited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 27; 2024: 23; 2025: 22; 2026: 16.

92 papers.

Sort:

By area:

  1. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  2. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  3. 2023Faithful Chain-of-Thought Reasoningcited by 288
  4. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
  5. 2024Aligner: Efficient Alignment by Learning to Correctcited by 98
  6. 2023Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanationscited by 85
  7. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  8. 2024LLM-based NLG Evaluation: Current Status and Challengescited by 73
  9. 2023Post Hoc Explanations of Language Models Can Improve Language Modelscited by 50
  10. 2023Explaining How Transformers Use Context to Build Predictionscited by 43
  11. 2023How are Prompts Different in Terms of Sensitivity?cited by 35
  12. 2023Interpreting Pretrained Language Models via Concept Bottleneckscited by 33
38 more
  1. 2023Analyzing Chain-of-Thought Prompting in Large Language Models via Gradient-based Feature Attributionscited by 32
  2. 2024MambaLRP: Explaining Selective State Space Sequence Modelscited by 32
  3. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  4. 2025Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistantcited by 26
  5. 2024Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generationcited by 25
  6. 2024On the Similarity of Circuits across Languages: a Case Study on the Subject-verb Agreement Taskcited by 24
  7. 2024AttriBoT: A Bag of Tricks for Efficiently Approximating Leave-One-Out Context Attributioncited by 23
  8. 2024Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agendacited by 22
  9. 2025QueryAttack: Jailbreaking Aligned Large Language Models Using Structured Non-natural Query Languagecited by 18
  10. 2023TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documentscited by 17
  11. 2025LinguaLens: Towards Interpreting Linguistic Mechanisms of Large Language Models via Sparse Auto-Encodercited by 14
  12. 2025Learning to Attribute with Attentioncited by 13
  13. 2023Surfacing Biases in Large Language Models using Contrastive Input Decodingcited by 11
  14. 2023Rather a Nurse than a Physician -- Contrastive Explanations under Investigationcited by 11
  15. 2023ALMANACS: A Simulatability Benchmark for Language Model Explainabilitycited by 11
  16. 2025A Close Look at Decomposition-based XAI-Methods for Transformer Language Modelscited by 11
  17. 2025The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentationcited by 11
  18. 2025On the Consistency of Multilingual Context Utilization in Retrieval-Augmented Generationcited by 10
  19. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  20. 2024Facilitating Human-LLM Collaboration through Factuality Scores and Source Attributionscited by 9
  21. 2023Feature Interactions Reveal Linguistic Structure in Language Modelscited by 8
  22. 2023Explaining with Contrastive Phrasal Highlighting: A Case Study in Assisting Humans to Detect Translation Differencescited by 8
  23. 2022Mediators: Conversational Agents Explaining NLP Model Behaviorcited by 7
  24. 2023Causal interventions expose implicit situation models for commonsense language understandingcited by 7
  25. 2023Quantifying the Plausibility of Context Reliance in Neural Machine Translationcited by 7
  26. 2023On the Interplay between Fairness and Explainabilitycited by 7
  27. 2024Interactive Prompt Debugging with Sequence Saliencecited by 6
  28. 2024Unveiling and Manipulating Prompt Influence in Large Language Modelscited by 6
  29. 2025Dagger Behind Smile: Fool LLMs with a Happy Ending Storycited by 6
  30. 2025GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMscited by 6
  31. 2023Faithfulness Measurable Masked Language Modelscited by 5
  32. 2024LLM Attributor: Interactive Visual Attribution for LLM Generationcited by 5
  33. 2024A Comparative Analysis of Counterfactual Explanation Methods for Text Classifierscited by 5
  34. 2024Attention with Dependency Parsing Augmentation for Fine-Grained Attributioncited by 5
  35. 2022Extend and Explain: Interpreting Very Long Language Modelscited by 4
  36. 2023Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learningcited by 4
  37. 2023Attribution and Alignment: Effects of Local Context Repetition on Utterance Production and Comprehension in Dialoguecited by 4
  38. 2023TaCo: Targeted Concept Erasure Prevents Non-Linear Classifiers From Detecting Protected Attributescited by 4

50 most cited of 92 All 92 (JSON)