# DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion

arXiv: https://arxiv.org/abs/1902.10526 · HTML: https://mlsanity.com/paper/1902.10526/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2019.

## Built on by (most cited first, 29 of 29)

- [CoEdIT: Text Editing by Task-Specific Instruction Tuning](https://mlsanity.com/paper/2305.09857/) (2023)
- [ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations](https://mlsanity.com/paper/2304.14827/) (2023)
- [EdiT5: Semi-Autoregressive Text-Editing with T5 Warm-Start](https://mlsanity.com/paper/2205.12209/) (2022)
- [Focus Attention: Promoting Faithfulness and Diversity in Summarization](https://mlsanity.com/paper/2105.11921/) (2021)
- [Improving Iterative Text Revision by Learning Where to Edit from Other Revision Tasks](https://mlsanity.com/paper/2212.01350/) (2022)
- [Neural Pipeline for Zero-Shot Data-to-Text Generation](https://mlsanity.com/paper/2203.16279/) (2022)
- [QA-Align: Representing Cross-Text Content Overlap by Aligning Question-Answer Propositions](https://mlsanity.com/paper/2109.12655/) (2021)
- [Summarization Programs: Interpretable Abstractive Summarization with Neural Modular Trees](https://mlsanity.com/paper/2209.10492/) (2022)
- [ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Models](https://mlsanity.com/paper/2210.04325/) (2022)
- [Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models](https://mlsanity.com/paper/2310.05861/) (2023)
- [Controlled Text Reduction](https://mlsanity.com/paper/2210.13449/) (2022)
- [Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning](https://mlsanity.com/paper/2208.02294/) (2022)
- [Non-autoregressive Text Editing with Copy-aware Latent Alignments](https://mlsanity.com/paper/2310.07821/) (2023)
- [The Power of Summary-Source Alignments](https://mlsanity.com/paper/2406.00842/) (2024)
- [Nutri-bullets: Summarizing Health Studies by Composing Segments](https://mlsanity.com/paper/2103.11921/) (2021)
- [Revisiting Sentence Union Generation as a Testbed for Text Consolidation](https://mlsanity.com/paper/2305.15605/) (2023)
- [Extending Multi-Text Sentence Fusion Resources via Pyramid Annotations](https://mlsanity.com/paper/2110.04517/) (2021)
- [Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operations](https://mlsanity.com/paper/2305.14599/) (2023)
- [Spivavtor: An Instruction Tuned Ukrainian Text Editing Model](https://mlsanity.com/paper/2404.18880/) (2024)
- [Patent-CR: A Dataset for Patent Claim Revision](https://mlsanity.com/paper/2412.02549/) (2024)
- [GenSco: Can Question Decomposition based Passage Alignment improve Question Answering?](https://mlsanity.com/paper/2407.10245/) (2024)
- [DiscoSense: Commonsense Reasoning with Discourse Connectives](https://mlsanity.com/paper/2210.12478/) (2022)
- [Conciseness: An Overlooked Language Task](https://mlsanity.com/paper/2211.04126/) (2022)
- [Multi-Review Fusion-in-Context](https://mlsanity.com/paper/2403.15351/) (2024)
- [Evaluating Discourse Cohesion in Pre-trained Language Models](https://mlsanity.com/paper/2503.06137/) (2025)
- [What Makes a Message Persuasive? Identifying Adaptations Towards Persuasiveness in Nine Exploratory Case Studies](https://mlsanity.com/paper/2104.12454/) (2021)
- [Comparative Snippet Generation](https://mlsanity.com/paper/2206.05473/) (2022)
- [G-SPEED: General SParse Efficient Editing MoDel](https://mlsanity.com/paper/2310.10480/) (2023)
- [KSOD: Knowledge Supplement for LLMs On Demand](https://mlsanity.com/paper/2503.07550/) (2025)

## Abstract (arXiv)

Sentence fusion is the task of joining several independent sentences into a single coherent text. Current datasets for sentence fusion are small and insufficient for training modern neural models. In this paper, we propose a method for automatically-generating fusion examples from raw text and present DiscoFuse, a large scale dataset for discourse-based sentence fusion. We author a set of rules for identifying a diverse set of discourse phenomena in raw text, and decomposing the text into two independent sentences. We apply our approach on two document collections: Wikipedia and Sports articles, yielding 60 million fusion examples annotated with discourse information required to reconstruct the fused text. We develop a sequence-to-sequence model on DiscoFuse and thoroughly analyze its strengths and weaknesses with respect to the various discourse phenomena, using both automatic as well as human evaluation. Finally, we conduct transfer learning experiments with WebSplit, a recent dataset for text simplification. We show that pretraining on DiscoFuse substantially improves performance on WebSplit when viewed as a sentence fusion task.