mlsanityData to 14 Oct 2026

Contrastive Attribution in the Wild: An Interpretability Analysis of LLM Failures on Realistic Benchmarks

arXiv 2604.17761 ↗First posted 20 Apr 2026cs.AI cs.CLcited by 2

On the map: Mechanistic interpretability → Knowledge and representations inside LLMs

Abstract (arXiv)

Interpretability tools are increasingly used to analyze failures of Large Language Models (LLMs), yet prior work largely focuses on short prompts or toy settings, leaving their behavior on commonly used benchmarks underexplored. To address this gap, we study contrastive, LRP-based attribution as a practical tool for analyzing LLM failures in realistic settings. We formulate failure analysis as textitcontrastive attribution, attributing the logit difference between an incorrect output token and a correct alternative to input tokens and internal model states, and introduce an efficient extension that enables construction of cross-layer attribution graphs for long-context inputs. Using this framework, we conduct a systematic empirical study across benchmarks, comparing attribution patterns across datasets, model sizes, and training checkpoints. Our results show that this token-level contrastive attribution can yield informative signals in some failure cases, but is not universally applicable, highlighting both its utility and its limitations for realistic LLM failure analysis. Our code is available at: https://aka.ms/Debug-XAI.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Knowledge and representations inside LLMs, one of 13 directions in Mechanistic interpretability. Builds on 51 of its 85 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

51 of 85 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Stream: Scaling up Mechanistic Interpretability to Long Context in LLMs via Sparse Attention22 Oct
  2. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methods27 Sep

Other references · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  9. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  10. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  11. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  12. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
37 more
  1. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  2. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  3. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  4. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  5. 2017SmoothGrad: removing noise by adding noisecited by 1,109
  6. 2022Mass-Editing Memory in a Transformercited by 1,001
  7. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  8. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  9. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 612
  10. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  11. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  12. 2020Explaining by Removing: A Unified Framework for Model Explanationcited by 218
  13. 2024How to use and interpret activation patchingcited by 214
  14. 2024Transcoders Find Interpretable LLM Feature Circuitscited by 200
  15. 2023Localizing Model Behavior with Path Patchingcited by 195
  16. 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systemscited by 185
  17. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 120
  18. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 111
  19. 2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100
  20. 2024Information Flow Routes: Automatically Interpreting Language Models at Scalecited by 95
  21. 2022Interpreting Language Models with Contrastive Explanationscited by 92
  22. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
  23. 2024Evaluating Language Models for Efficient Code Generationcited by 73
  24. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  25. 2024Spectral Filters, Dark Signals, and Attention Sinkscited by 56
  26. 2021Incorporating Residual and Normalization Layers into Analysis of Masked Language Modelscited by 51
  27. 2014Extraction of Salient Sentences from Labelled Documentscited by 50
  28. 2024From Understanding to Utilization: A Survey on Explainability for Large Language Modelscited by 48
  29. 2025I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoderscited by 45
  30. 2023Explaining How Transformers Use Context to Build Predictionscited by 43
  31. 2024AttriBoT: A Bag of Tricks for Efficiently Approximating Leave-One-Out Context Attributioncited by 23
  32. 2025Verifying Chain-of-Thought Reasoning via Its Computational Graphcited by 20
  33. 2025Token-Importance Guided Direct Preference Optimizationcited by 14
  34. 2025A Close Look at Decomposition-based XAI-Methods for Transformer Language Modelscited by 11
  35. 2025From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Modelscited by 6
  36. 2026ErrorMap and ErrorAtlas: Charting the Failure Landscape of Large Language Modelscited by 6
  37. 2025Diagnosing Failures in Large Language Models' Answers: Integrating Error Attribution into Evaluation Frameworkcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Auditing Agent Actions through Query-Conditioned Attributioncited by 0
  2. 2026RoPE at the End of Its Rope? Theory, Diagnosis, and Mitigation of Long-Context Failurescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Is my model perplexed for the right reason? Contrasting LLMs' Benchmark Behavior with Token-Level Perplexity
  2. 2025Where Did It Go Wrong? Attributing Undesirable LLM Behaviors via Representation Gradient Tracing
  3. 2026How Language Models Fail: Token-Level Signatures of Committed and Persistent Reasoning Failures
  4. 2026LLM attribution analysis across different fine-tuning strategies and model scales for automated code compliance
  5. 2025Can LLMs Detect Their Confabulations? Estimating Reliability in Uncertainty-Aware Language Models
  6. 2026Towards Long-Horizon Interpretability: Efficient and Faithful Multi-Token Attribution for Reasoning LLMs
  7. 2026Context Dependence and Reliability in Autoregressive Language Models
  8. 2026Detecting Inconsistencies in Model Specifications with LLM-as-Verifier Reasoning