# A Rate-Distortion Framework for Summarization

arXiv: https://arxiv.org/abs/2501.13100 · HTML: https://mlsanity.com/paper/2501.13100/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 22 Jan 2025.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models](https://mlsanity.com/paper/2310.05736/) (2023)
- [Re-evaluating Evaluation in Text Summarization](https://mlsanity.com/paper/2010.07100/) (2020)
- [FineSurE: Fine-grained Summarization Evaluation using LLMs](https://mlsanity.com/paper/2407.00908/) (2024)
- [Mutual Information Alleviates Hallucinations in Abstractive Summarization](https://mlsanity.com/paper/2210.13210/) (2022)
- [Mutual Information Divergence: A Unified Metric for Multimodal Generative Models](https://mlsanity.com/paper/2205.13445/) (2022)
- [Unsupervised Extractive Summarization using Pointwise Mutual Information](https://mlsanity.com/paper/2102.06272/) (2021)
- [Fundamental Limits of Prompt Compression: A Rate-Distortion Framework for Black-Box Language Models](https://mlsanity.com/paper/2407.15504/) (2024)
- [textttCOSMIC: Mutual Information for Task-Agnostic Summarization Evaluation](https://mlsanity.com/paper/2402.19457/) (2024)

## Built on by (most cited first, 3 of 3)

- [An Information Theoretic Perspective on Agentic System Design](https://mlsanity.com/paper/2512.21720/) (2025)
- [Fixed RAG Compression Collapses Measured Reader Scaling](https://mlsanity.com/paper/2606.21807/) (2026)
- [The Hidden Perception Constraint in Task-Aware Compression](https://mlsanity.com/paper/2609.35684/) (2026)

## Abstract (arXiv)

This paper introduces an information-theoretic framework for text summarization. We define the summarizer rate-distortion function and show that it provides a fundamental lower bound on summarizer performance. We describe an iterative procedure, similar to Blahut-Arimoto algorithm, for computing this function. To handle real-world text datasets, we also propose a practical method that can calculate the summarizer rate-distortion function with limited data. Finally, we empirically confirm our theoretical results by comparing the summarizer rate-distortion function with the performances of different summarizers used in practice.