# WikiSplit++: Easy Data Refinement for Split and Rephrase

arXiv: https://arxiv.org/abs/2404.09002 · HTML: https://mlsanity.com/paper/2404.09002/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 13 Apr 2024.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [Six Challenges for Neural Machine Translation](https://mlsanity.com/paper/1706.03872/) (2017)
- [Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering](https://mlsanity.com/paper/2104.08202/) (2021)
- [BLEU is Not Suitable for the Evaluation of Text Simplification](https://mlsanity.com/paper/1810.05995/) (2018)
- [Controllable Sentence Simplification](https://mlsanity.com/paper/1910.02677/) (2019)
- [EASSE: Easier Automatic Sentence Simplification Evaluation](https://mlsanity.com/paper/1908.04567/) (2019)
- [Learning To Split and Rephrase From Wikipedia Edit History](https://mlsanity.com/paper/1808.09468/) (2018)
- [PySBD: Pragmatic Sentence Boundary Disambiguation](https://mlsanity.com/paper/2010.09657/) (2020)
- [Neural Machine Translation with Reconstruction](https://mlsanity.com/paper/1611.01874/) (2016)
- [Split and Rephrase](https://mlsanity.com/paper/1707.06971/) (2017)
- [BiSECT: Learning to Split and Rephrase Sentences with Bitexts](https://mlsanity.com/paper/2109.05006/) (2021)
- [Split and Rephrase: Better Evaluation and a Stronger Baseline](https://mlsanity.com/paper/1805.01035/) (2018)
- [Transforming Complex Sentences into a Semantic Hierarchy](https://mlsanity.com/paper/1906.01038/) (2019)
- [Overcoming the Curse of Sentence Length for Neural Machine Translation using Automatic Segmentation](https://mlsanity.com/paper/1409.1257/) (2014)
- [MinWikiSplit: A Sentence Splitting Corpus with Minimal Propositions](https://mlsanity.com/paper/1909.12131/) (2019)
- [Small but Mighty: New Benchmarks for Split and Rephrase](https://mlsanity.com/paper/2009.08560/) (2020)

## Built on by (most cited first, 2 of 2)

- [CREDENCE: Claim Reduction for Decomposition & Enhanced Credibility -- Semantic Metrics and Convergence Analysis](https://mlsanity.com/paper/2606.19819/) (2026)
- [KSOD: Knowledge Supplement for LLMs On Demand](https://mlsanity.com/paper/2503.07550/) (2025)

## Abstract (arXiv)

The task of Split and Rephrase, which splits a complex sentence into multiple simple sentences with the same meaning, improves readability and enhances the performance of downstream tasks in natural language processing (NLP). However, while Split and Rephrase can be improved using a text-to-text generation approach that applies encoder-decoder models fine-tuned with a large-scale dataset, it still suffers from hallucinations and under-splitting. To address these issues, this paper presents a simple and strong data refinement approach. Here, we create WikiSplit++ by removing instances in WikiSplit where complex sentences do not entail at least one of the simpler sentences and reversing the order of reference simple sentences. Experimental results show that training with WikiSplit++ leads to better performance than training with WikiSplit, even with fewer training instances. In particular, our approach yields significant gains in the number of splits and the entailment ratio, a proxy for measuring hallucinations.