# Keep it Simple: Unsupervised Simplification of Multi-Paragraph Text

arXiv: https://arxiv.org/abs/2107.03444 · HTML: https://mlsanity.com/paper/2107.03444/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 53 references that are in the map.
- First posted on arXiv 7 Jul 2021.

## Builds on (most cited first, 19 of 19)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016)
- [Asking and Answering Questions to Evaluate the Factual Consistency of Summaries](https://mlsanity.com/paper/2004.04228/) (2020)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [Sentence Simplification with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.10931/) (2017)
- [Factual Error Correction for Abstractive Summarization Models](https://mlsanity.com/paper/2010.08712/) (2020)
- [Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports](https://mlsanity.com/paper/1911.02541/) (2019)
- [Controllable Sentence Simplification](https://mlsanity.com/paper/1910.02677/) (2019)
- [What Have We Achieved on Text Summarization?](https://mlsanity.com/paper/2010.04529/) (2020)
- [EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing](https://mlsanity.com/paper/1906.08104/) (2019)
- [Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering](https://mlsanity.com/paper/1909.06356/) (2019)
- [Seq2Edits: Sequence Transduction Using Span-level Edit Operations](https://mlsanity.com/paper/2009.11136/) (2020)
- [Deep Communicating Agents for Abstractive Summarization](https://mlsanity.com/paper/1803.10357/) (2018)
- [Sentence-Level Fluency Evaluation: References Help, But Can Be Spared!](https://mlsanity.com/paper/1809.08731/) (2018)
- [Unsupervised Neural Text Simplification](https://mlsanity.com/paper/1810.07931/) (2018)
- [The Summary Loop: Learning to Write Abstractive Summaries Without Examples](https://mlsanity.com/paper/2105.05361/) (2021)
- [Dynamic Multi-Level Multi-Task Learning for Sentence Simplification](https://mlsanity.com/paper/1806.07304/) (2018)
- [Discourse Level Factors for Sentence Deletion in Text Simplification](https://mlsanity.com/paper/1911.10384/) (2019)

## Built on by (most cited first, 30 of 45)

- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [ChatGPT as a Factual Inconsistency Evaluator for Text Summarization](https://mlsanity.com/paper/2303.15621/) (2023)
- [Graphologue: Exploring Large Language Model Responses with Interactive Diagrams](https://mlsanity.com/paper/2305.11473/) (2023)
- [Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference](https://mlsanity.com/paper/2409.01227/) (2024)
- [Discrete Prompt Compression with Reinforcement Learning](https://mlsanity.com/paper/2308.08758/) (2023)
- [Beyond the Chat: Executable and Verifiable Text-Editing with LLMs](https://mlsanity.com/paper/2309.15337/) (2023)
- [Evaluating Factuality in Text Simplification](https://mlsanity.com/paper/2204.07562/) (2022)
- [A Better Way to Do Masked Language Model Scoring](https://mlsanity.com/paper/2305.10588/) (2023)
- [Generating Summaries with Controllable Readability Levels](https://mlsanity.com/paper/2310.10623/) (2023)
- [Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processing](https://mlsanity.com/paper/2203.00130/) (2022)
- [Zero-shot Faithfulness Evaluation for Text Summarization with Foundation Language Model](https://mlsanity.com/paper/2310.11648/) (2023)
- [Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning](https://mlsanity.com/paper/2205.08221/) (2022)
- [Do Text Simplification Systems Preserve Meaning? A Human Evaluation via Reading Comprehension](https://mlsanity.com/paper/2312.10126/) (2023)
- [Dancing Between Success and Failure: Edit-level Simplification Evaluation using SALSA](https://mlsanity.com/paper/2305.14458/) (2023)
- [SWiPE: A Dataset for Document-Level Simplification of Wikipedia Pages](https://mlsanity.com/paper/2305.19204/) (2023)
- [SciNews: From Scholarly Complexities to Public Narratives -- A Dataset for Scientific News Report Generation](https://mlsanity.com/paper/2403.17768/) (2024)
- [Authorship Style Transfer with Policy Optimization](https://mlsanity.com/paper/2403.08043/) (2024)
- [Teaching the Pre-trained Model to Generate Simple Texts for Text Simplification](https://mlsanity.com/paper/2305.12463/) (2023)
- [Dynamic Reward Adjustment in Multi-Reward Reinforcement Learning for Counselor Reflection Generation](https://mlsanity.com/paper/2403.13578/) (2024)
- [Keep It Private: Unsupervised Privatization of Online Text](https://mlsanity.com/paper/2405.10260/) (2024)
- [An Empirical Study on Prompt Compression for Large Language Models](https://mlsanity.com/paper/2505.00019/) (2025)
- [Exploiting Summarization Data to Help Text Simplification](https://mlsanity.com/paper/2302.07124/) (2023)
- [An LLM-Enhanced Adversarial Editing System for Lexical Simplification](https://mlsanity.com/paper/2402.14704/) (2024)
- [Evaluating Document Simplification: On the Importance of Separately Assessing Simplicity and Meaning Preservation](https://mlsanity.com/paper/2404.03278/) (2024)
- [TAROT: Task-Oriented Authorship Obfuscation Using Policy Optimization Methods](https://mlsanity.com/paper/2407.21630/) (2024)
- [Progressive Document-level Text Simplification via Large Language Models](https://mlsanity.com/paper/2501.03857/) (2025)
- [Steering Large Language Models with Register Analysis for Arbitrary Style Transfer](https://mlsanity.com/paper/2505.00679/) (2025)
- [Automatic and Human-AI Interactive Text Generation](https://mlsanity.com/paper/2310.03878/) (2023)
- [Style Transfer with Multi-iteration Preference Optimization](https://mlsanity.com/paper/2406.11581/) (2024)

## Abstract (arXiv)

This work presents Keep it Simple (KiS), a new approach to unsupervised text simplification which learns to balance a reward across three properties: fluency, salience and simplicity. We train the model with a novel algorithm to optimize the reward (k-SCST), in which the model proposes several candidate simplifications, computes each candidate's reward, and encourages candidates that outperform the mean reward. Finally, we propose a realistic text comprehension task as an evaluation method for text simplification. When tested on the English news domain, the KiS model outperforms strong supervised baselines by more than 4 SARI points, and can help people complete a comprehension task an average of 18% faster while retaining accuracy, when compared to the original text. Code available: https://github.com/tingofurro/keep_it_simple