# Unsupervised Extractive Summarization using Pointwise Mutual Information

arXiv: https://arxiv.org/abs/2102.06272 · HTML: https://mlsanity.com/paper/2102.06272/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 11 Feb 2021.

## Builds on (most cited first, 13 of 13)

- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents](https://mlsanity.com/paper/1611.04230/) (2016)
- [Abstractive Summarization of Reddit Posts with Multi-level Memory Networks](https://mlsanity.com/paper/1811.00783/) (2018)
- [Sentence Centrality Revisited for Unsupervised Summarization](https://mlsanity.com/paper/1906.03508/) (2019)
- [Content Selection in Deep Learning Models of Summarization](https://mlsanity.com/paper/1810.12343/) (2018)
- [The price of debiasing automatic metrics in natural language evaluation](https://mlsanity.com/paper/1807.02202/) (2018)
- [BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle](https://mlsanity.com/paper/1909.07405/) (2019)
- [Variations of the Similarity Function of TextRank for Automated Summarization](https://mlsanity.com/paper/1602.03606/) (2016)
- [Simple Unsupervised Summarization by Contextual Matching](https://mlsanity.com/paper/1907.13337/) (2019)
- [Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers](https://mlsanity.com/paper/2010.08242/) (2020)

## Built on by (most cited first, 23 of 23)

- [Evaluating the Factual Consistency of Large Language Models Through News Summarization](https://mlsanity.com/paper/2211.08412/) (2022)
- [Pre-trained Language Models in Biomedical Domain: A Systematic Survey](https://mlsanity.com/paper/2110.05006/) (2021)
- [ReCEval: Evaluating Reasoning Chains via Correctness and Informativeness](https://mlsanity.com/paper/2304.10703/) (2023)
- [Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics](https://mlsanity.com/paper/2204.10216/) (2022)
- [Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarization](https://mlsanity.com/paper/2209.03549/) (2022)
- [InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding](https://mlsanity.com/paper/2306.04933/) (2023)
- [OTExtSum: Extractive Text Summarisation with Optimal Transport](https://mlsanity.com/paper/2204.10086/) (2022)
- [GraphNarrator: Generating Textual Explanations for Graph Neural Networks](https://mlsanity.com/paper/2410.15268/) (2024)
- [textttCOSMIC: Mutual Information for Task-Agnostic Summarization Evaluation](https://mlsanity.com/paper/2402.19457/) (2024)
- [Information-Theoretic Distillation for Reference-less Summarization](https://mlsanity.com/paper/2403.13780/) (2024)
- [Unsupervised Multi-Granularity Summarization](https://mlsanity.com/paper/2201.12502/) (2022)
- [Understanding Position Bias Effects on Fairness in Social Multi-Document Summarization](https://mlsanity.com/paper/2405.01790/) (2024)
- [Achieving Conversational Goals with Unsupervised Post-hoc Knowledge Injection](https://mlsanity.com/paper/2203.11399/) (2022)
- [A Rate-Distortion Framework for Summarization](https://mlsanity.com/paper/2501.13100/) (2025)
- [Source Identification in Abstractive Summarization](https://mlsanity.com/paper/2402.04677/) (2024)
- [RTSUM: Relation Triple-based Interpretable Summarization with Multi-level Salience Visualization](https://mlsanity.com/paper/2310.13895/) (2023)
- [Unsupervised Extractive Summarization with Learnable Length Control Strategies](https://mlsanity.com/paper/2312.06901/) (2023)
- [Enhancing Long Document Long Form Summarisation with Self-Planning](https://mlsanity.com/paper/2512.17179/) (2025)
- [Summary Workbench: Unifying Application and Evaluation of Text Summarization Models](https://mlsanity.com/paper/2210.09587/) (2022)
- [Unsupervised Scientific Abstract Segmentation with Normalized Mutual Information](https://mlsanity.com/paper/2305.11553/) (2023)
- [Efficient and Interpretable Compressive Text Summarisation with Unsupervised Dual-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.03415/) (2023)
- [Which Information Matters? Dissecting Human-written Multi-document Summaries with Partial Information Decomposition](https://mlsanity.com/paper/2405.14470/) (2024)
- [Scoring Rules! Statistical and Strategic Alignment for Text Evaluation Metrics](https://mlsanity.com/paper/2608.01423/) (2026)

## Abstract (arXiv)

Unsupervised approaches to extractive summarization usually rely on a notion of sentence importance defined by the semantic similarity between a sentence and the document. We propose new metrics of relevance and redundancy using pointwise mutual information (PMI) between sentences, which can be easily computed by a pre-trained language model. Intuitively, a relevant sentence allows readers to infer the document content (high PMI with the document), and a redundant sentence can be inferred from the summary (high PMI with the summary). We then develop a greedy sentence selection algorithm to maximize relevance and minimize redundancy of extracted sentences. We show that our method outperforms similarity-based methods on datasets in a range of domains including news, medical journal articles, and personal anecdotes.