mlsanityData to 7 Oct 2026

Multi-Level Explanations for Generative Language Models

arXiv 2403.14459 ↗First posted 21 Mar 2024cs.CL cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Despite the increasing use of large language models (LLMs) for context-grounded tasks like summarization and question-answering, understanding what makes an LLM produce a certain response is challenging. We propose Multi-Level Explanations for Generative Language Models (MExGen), a technique to provide explanations for context-grounded text generation. MExGen assigns scores to parts of the context to quantify their influence on the model's output. It extends attribution methods like LIME and SHAP to LLMs used in context-grounded tasks where (1) inference cost is high, (2) input text is long, and (3) the output is text. We conduct a systematic evaluation, both automated and human, of perturbation-based attribution methods for summarization and question answering. The results show that our framework can provide more faithful explanations of generated output than available alternatives, including LLM self-explanations. We open-source code for MExGen as part of the ICX360 toolkit: https://github.com/IBM/ICX360.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 41 of its 64 references that are in the map. First posted on arXiv 21 Mar 2024.

Builds on

41 of 64 references.

References · 41

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  9. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  10. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  11. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  12. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
29 more
  1. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  2. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  3. 2015Teaching Machines to Read and Comprehendcited by 1,039
  4. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  5. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  6. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  7. 2022UL2: Unifying Language Learning Paradigmscited by 292
  8. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  9. 2024Are self-explanations from Large Language Models faithful?cited by 101
  10. 2018L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Datacited by 100
  11. 2023Can Large Language Models Explain Themselves? A Study of LLM-Generated Self-Explanationscited by 90
  12. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  13. 2023On Measuring Faithfulness or Self-consistency of Natural Language Explanationscited by 71
  14. 2018Hierarchical interpretations for neural network predictionscited by 66
  15. 2020Generating Hierarchical Explanations on Text Classification via Feature Interaction Detectioncited by 57
  16. 2020Interpretation of NLP models through input marginalizationcited by 43
  17. 2023Building Real-World Meeting Summarization Systems using Large Language Models: A Practical Perspectivecited by 34
  18. 2023Using Captum to Explain Generative Language Modelscited by 33
  19. 2019Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Modelscited by 26
  20. 2020An Analysis of LIME for Text Datacited by 25
  21. 2023D-Separation for Causal Self-Explanationcited by 21
  22. 2023GLIME: General, Stable and Local LIME Explanationcited by 21
  23. 2022FR: Folded Rationalization with a Unified Encodercited by 18
  24. 2023TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documentscited by 17
  25. 2023Enhancing the Rationale-Input Alignment for Self-explaining Rationalizationcited by 15
  26. 2023Decoupled Rationalization with Asymmetric Learning Rates: A Flexible Lipschitz Restraintcited by 14
  27. 2023SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Modelscited by 14
  28. 2019LS-Tree: Model Interpretation When the Data Are Linguisticcited by 7
  29. 2021Double Trouble: How to not explain a text classifier's decisions using counterfactuals synthesized by masked language models?cited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 6; 2026: 3.

11 papers.

  1. 2025SPEX: Scaling Feature Interaction Explanations for LLMscited by 12
  2. 2025Explaining the Reasoning of Large Language Models Using Attribution Graphscited by 3
  3. 2024Selective Explanationscited by 2
  4. 2025Revitalizing Black-Box Interpretability: Actionable Interpretability for LLMs via Proxy Modelscited by 2
  5. 2025Context Attribution with Multi-Armed Bandit Optimizationcited by 2
  6. 2025Hide or Highlight: Understanding the Impact of Factuality Expression on User Trustcited by 1
  7. 2024CELL your Model: Contrastive Explanations for Large Language Modelscited by 0
  8. 2025Attribution assignment for deep-generative sequence models enables interpretability analysis using positive-only datacited by 0
  9. 2026LIME-LLM: Probing Models with Fluent Counterfactuals, Not Broken Textcited by 0
  10. 2026Re-Ranking Through an Attribution Lens for Citation Quality in Legal QAcited by 0
  11. 2026Topological Attribution Distance (TAD): Revealing Segment-Level RAG Influence on LLM Output Geometry for Incident Log Analysiscited by 0