# Simple Unsupervised Summarization by Contextual Matching

arXiv: https://arxiv.org/abs/1907.13337 · HTML: https://mlsanity.com/paper/1907.13337/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2019.

## Built on by (most cited first, 18 of 18)

- [A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods](https://mlsanity.com/paper/2403.02901/) (2024)
- [Discrete Prompt Compression with Reinforcement Learning](https://mlsanity.com/paper/2308.08758/) (2023)
- [ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining](https://mlsanity.com/paper/2106.00829/) (2021)
- [Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing](https://mlsanity.com/paper/2305.16635/) (2023)
- [AMR Parsing with Action-Pointer Transformer](https://mlsanity.com/paper/2104.14674/) (2021)
- [Unsupervised Extractive Summarization using Pointwise Mutual Information](https://mlsanity.com/paper/2102.06272/) (2021)
- [Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning](https://mlsanity.com/paper/2205.08221/) (2022)
- [Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization](https://mlsanity.com/paper/2205.14521/) (2022)
- [LawSum: A weakly supervised approach for Indian Legal Document Summarization](https://mlsanity.com/paper/2110.01188/) (2021)
- [Generating Multiple-Length Summaries via Reinforcement Learning for Unsupervised Sentence Summarization](https://mlsanity.com/paper/2212.10843/) (2022)
- [An Efficient Coarse-to-Fine Facet-Aware Unsupervised Summarization Framework based on Semantic Blocks](https://mlsanity.com/paper/2208.08253/) (2022)
- [InstructCMP: Length Control in Sentence Compression through Instruction-based Large Language Models](https://mlsanity.com/paper/2406.11097/) (2024)
- [With Measured Words: Simple Sentence Selection for Black-Box Optimization of Sentence Compression Algorithms](https://mlsanity.com/paper/2101.10096/) (2021)
- [Evaluation of Abstractive Summarisation Models with Machine Translation in Deliberative Processes](https://mlsanity.com/paper/2110.05847/) (2021)
- [Recent Trends in Unsupervised Summarization](https://mlsanity.com/paper/2305.11231/) (2023)
- [Unsupervised Open-domain Keyphrase Generation](https://mlsanity.com/paper/2306.10755/) (2023)
- [From Lengthy to Lucid: A Systematic Literature Review on NLP Techniques for Taming Long Sentences](https://mlsanity.com/paper/2312.05172/) (2023)
- [SinBrief: A Hybrid Framework for Abstractive Text Summarisation of Sinhala Legal Documents](https://mlsanity.com/paper/2609.32397/) (2026)

## Abstract (arXiv)

We propose an unsupervised method for sentence summarization using only language modeling. The approach employs two language models, one that is generic (i.e. pretrained), and the other that is specific to the target domain. We show that by using a product-of-experts criteria these are enough for maintaining continuous contextual matching while maintaining output fluency. Experiments on both abstractive and extractive sentence summarization data sets show promising results of our method without being exposed to any paired data.