# Multi-Level Explanations for Generative Language Models

arXiv: https://arxiv.org/abs/2403.14459 · HTML: https://mlsanity.com/paper/2403.14459/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 64 references that are in the map.
- First posted on arXiv 21 Mar 2024.

## Builds on (most cited first, 30 of 41)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [Benchmarking Retrieval-Augmented Generation for Medicine](https://mlsanity.com/paper/2402.13178/) (2024)
- [UL2: Unifying Language Learning Paradigms](https://mlsanity.com/paper/2205.05131/) (2022)
- [Faithfulness Tests for Natural Language Explanations](https://mlsanity.com/paper/2305.18029/) (2023)
- [Are self-explanations from Large Language Models faithful?](https://mlsanity.com/paper/2401.07927/) (2024)
- [L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Data](https://mlsanity.com/paper/1808.02610/) (2018)
- [Can Large Language Models Explain Themselves? A Study of LLM-Generated Self-Explanations](https://mlsanity.com/paper/2310.11207/) (2023)
- [Inseq: An Interpretability Toolkit for Sequence Generation Models](https://mlsanity.com/paper/2302.13942/) (2023)
- [On Measuring Faithfulness or Self-consistency of Natural Language Explanations](https://mlsanity.com/paper/2311.07466/) (2023)
- [Hierarchical interpretations for neural network predictions](https://mlsanity.com/paper/1806.05337/) (2018)
- [Generating Hierarchical Explanations on Text Classification via Feature Interaction Detection](https://mlsanity.com/paper/2004.02015/) (2020)
- [Interpretation of NLP models through input marginalization](https://mlsanity.com/paper/2010.13984/) (2020)
- [Building Real-World Meeting Summarization Systems using Large Language Models: A Practical Perspective](https://mlsanity.com/paper/2310.19233/) (2023)
- [Using Captum to Explain Generative Language Models](https://mlsanity.com/paper/2312.05491/) (2023)

## Built on by (most cited first, 11 of 11)

- [SPEX: Scaling Feature Interaction Explanations for LLMs](https://mlsanity.com/paper/2502.13870/) (2025)
- [Explaining the Reasoning of Large Language Models Using Attribution Graphs](https://mlsanity.com/paper/2512.15663/) (2025)
- [Selective Explanations](https://mlsanity.com/paper/2405.19562/) (2024)
- [Revitalizing Black-Box Interpretability: Actionable Interpretability for LLMs via Proxy Models](https://mlsanity.com/paper/2505.12509/) (2025)
- [Context Attribution with Multi-Armed Bandit Optimization](https://mlsanity.com/paper/2506.19977/) (2025)
- [Hide or Highlight: Understanding the Impact of Factuality Expression on User Trust](https://mlsanity.com/paper/2508.07095/) (2025)
- [CELL your Model: Contrastive Explanations for Large Language Models](https://mlsanity.com/paper/2406.11785/) (2024)
- [Attribution assignment for deep-generative sequence models enables interpretability analysis using positive-only data](https://mlsanity.com/paper/2506.23182/) (2025)
- [LIME-LLM: Probing Models with Fluent Counterfactuals, Not Broken Text](https://mlsanity.com/paper/2601.11746/) (2026)
- [Re-Ranking Through an Attribution Lens for Citation Quality in Legal QA](https://mlsanity.com/paper/2606.03728/) (2026)
- [Topological Attribution Distance (TAD): Revealing Segment-Level RAG Influence on LLM Output Geometry for Incident Log Analysis](https://mlsanity.com/paper/2608.16775/) (2026)

## Abstract (arXiv)

Despite the increasing use of large language models (LLMs) for context-grounded tasks like summarization and question-answering, understanding what makes an LLM produce a certain response is challenging. We propose Multi-Level Explanations for Generative Language Models (MExGen), a technique to provide explanations for context-grounded text generation. MExGen assigns scores to parts of the context to quantify their influence on the model's output. It extends attribution methods like LIME and SHAP to LLMs used in context-grounded tasks where (1) inference cost is high, (2) input text is long, and (3) the output is text. We conduct a systematic evaluation, both automated and human, of perturbation-based attribution methods for summarization and question answering. The results show that our framework can provide more faithful explanations of generated output than available alternatives, including LLM self-explanations. We open-source code for MExGen as part of the ICX360 toolkit: https://github.com/IBM/ICX360.