# Small but Mighty: New Benchmarks for Split and Rephrase

arXiv: https://arxiv.org/abs/2009.08560 · HTML: https://mlsanity.com/paper/2009.08560/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2020.

## Built on by (most cited first, 8 of 8)

- [GEMv2: Multilingual NLG Benchmarking in a Single Line of Code](https://mlsanity.com/paper/2206.11249/) (2022)
- [BiSECT: Learning to Split and Rephrase Sentences with Bitexts](https://mlsanity.com/paper/2109.05006/) (2021)
- [Paraphrase Detection: Human vs. Machine Content](https://mlsanity.com/paper/2303.13989/) (2023)
- [Rewriting the Script: Adapting Text Instructions for Voice Interaction](https://mlsanity.com/paper/2306.09992/) (2023)
- [Split and Rephrase with Large Language Models](https://mlsanity.com/paper/2312.11075/) (2023)
- [WikiSplit++: Easy Data Refinement for Split and Rephrase](https://mlsanity.com/paper/2404.09002/) (2024)
- [Syntactic Complexity Identification, Measurement, and Reduction Through Controlled Syntactic Simplification](https://mlsanity.com/paper/2304.07774/) (2023)
- [From Lengthy to Lucid: A Systematic Literature Review on NLP Techniques for Taming Long Sentences](https://mlsanity.com/paper/2312.05172/) (2023)

## Abstract (arXiv)

Split and Rephrase is a text simplification task of rewriting a complex sentence into simpler ones. As a relatively new task, it is paramount to ensure the soundness of its evaluation benchmark and metric. We find that the widely used benchmark dataset universally contains easily exploitable syntactic cues caused by its automatic generation process. Taking advantage of such cues, we show that even a simple rule-based model can perform on par with the state-of-the-art model. To remedy such limitations, we collect and release two crowdsourced benchmark datasets. We not only make sure that they contain significantly more diverse syntax, but also carefully control for their quality according to a well-defined set of criteria. While no satisfactory automatic metric exists, we apply fine-grained manual evaluation based on these criteria using crowdsourcing, showing that our datasets better represent the task and are significantly more challenging for the models.