# Unsupervised Sentence Compression using Denoising Auto-Encoders

arXiv: https://arxiv.org/abs/1809.02669 · HTML: https://mlsanity.com/paper/1809.02669/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Sep 2018.

## Built on by (most cited first, 15 of 15)

- [Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing](https://mlsanity.com/paper/2305.16635/) (2023)
- [Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning](https://mlsanity.com/paper/2205.08221/) (2022)
- [Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Survey](https://mlsanity.com/paper/2311.09212/) (2023)
- [Unsupervised Abstractive Opinion Summarization by Generating Sentences with Tree-Structured Topic Guidance](https://mlsanity.com/paper/2106.08007/) (2021)
- [Text Compression-aided Transformer Encoding](https://mlsanity.com/paper/2102.05951/) (2021)
- [Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization](https://mlsanity.com/paper/2205.14521/) (2022)
- [MMTAfrica: Multilingual Machine Translation for African Languages](https://mlsanity.com/paper/2204.04306/) (2022)
- [Information-Theoretic Distillation for Reference-less Summarization](https://mlsanity.com/paper/2403.13780/) (2024)
- [Generating Multiple-Length Summaries via Reinforcement Learning for Unsupervised Sentence Summarization](https://mlsanity.com/paper/2212.10843/) (2022)
- [EASE: Extractive-Abstractive Summarization with Explanations](https://mlsanity.com/paper/2105.06982/) (2021)
- [InstructCMP: Length Control in Sentence Compression through Instruction-based Large Language Models](https://mlsanity.com/paper/2406.11097/) (2024)
- [Bag-of-Vectors Autoencoders for Unsupervised Conditional Text Generation](https://mlsanity.com/paper/2110.07002/) (2021)
- [GLIMMER: Incorporating Graph and Lexical Features in Unsupervised Multi-Document Summarization](https://mlsanity.com/paper/2408.10115/) (2024)
- [Multi-Narrative Semantic Overlap Task: Evaluation and Benchmark](https://mlsanity.com/paper/2201.05294/) (2022)
- [From Lengthy to Lucid: A Systematic Literature Review on NLP Techniques for Taming Long Sentences](https://mlsanity.com/paper/2312.05172/) (2023)

## Abstract (arXiv)

In sentence compression, the task of shortening sentences while retaining the original meaning, models tend to be trained on large corpora containing pairs of verbose and compressed sentences. To remove the need for paired corpora, we emulate a summarization task and add noise to extend sentences and train a denoising auto-encoder to recover the original, constructing an end-to-end training regime without the need for any examples of compressed sentences. We conduct a human evaluation of our model on a standard text summarization dataset and show that it performs comparably to a supervised baseline based on grammatical correctness and retention of meaning. Despite being exposed to no target data, our unsupervised models learn to generate imperfect but reasonably readable sentence summaries. Although we underperform supervised models based on ROUGE scores, our models are competitive with a supervised baseline based on human evaluation for grammatical correctness and retention of meaning.