# ParaSCI: A Large Scientific Paraphrase Dataset for Longer Paraphrase Generation

arXiv: https://arxiv.org/abs/2101.08382 · HTML: https://mlsanity.com/paper/2101.08382/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 21 Jan 2021.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [A Continuously Growing Dataset of Sentential Paraphrases](https://mlsanity.com/paper/1708.00391/) (2017)
- [A Deep Generative Framework for Paraphrase Generation](https://mlsanity.com/paper/1709.05074/) (2017)
- [Learning to Paraphrase for Question Answering](https://mlsanity.com/paper/1708.06022/) (2017)
- [Paraphrase Generation with Latent Bag of Words](https://mlsanity.com/paper/2001.01941/) (2020)
- [Stochastic Answer Networks for Machine Reading Comprehension](https://mlsanity.com/paper/1712.03556/) (2017)
- [Cross-domain Semantic Parsing via Paraphrasing](https://mlsanity.com/paper/1704.05974/) (2017)

## Built on by (most cited first, 28 of 28)

- [Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing](https://mlsanity.com/paper/2305.16635/) (2023)
- [EditEval: An Instruction-Based Benchmark for Text Improvements](https://mlsanity.com/paper/2209.13331/) (2022)
- [arXivEdits: Understanding the Human Revision Process in Scientific Writing](https://mlsanity.com/paper/2210.15067/) (2022)
- [Novelty Controlled Paraphrase Generation with Retrieval Augmented Conditional Prompt Tuning](https://mlsanity.com/paper/2202.00535/) (2022)
- [A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Texts](https://mlsanity.com/paper/2311.08374/) (2023)
- [SciXGen: A Scientific Paper Dataset for Context-Aware Text Generation](https://mlsanity.com/paper/2110.10774/) (2021)
- [Towards Process-Oriented, Modular, and Versatile Question Generation that Meets Educational Needs](https://mlsanity.com/paper/2205.00355/) (2022)
- [The ACL OCL Corpus: Advancing Open Science in Computational Linguistics](https://mlsanity.com/paper/2305.14996/) (2023)
- [Neural semi-Markov CRF for Monolingual Word Alignment](https://mlsanity.com/paper/2106.02569/) (2021)
- [SsciBERT: A Pre-trained Language Model for Social Science Texts](https://mlsanity.com/paper/2206.04510/) (2022)
- [Paraphrase Detection: Human vs. Machine Content](https://mlsanity.com/paper/2303.13989/) (2023)
- [SciNews: From Scholarly Complexities to Public Narratives -- A Dataset for Scientific News Report Generation](https://mlsanity.com/paper/2403.17768/) (2024)
- [Finnish Paraphrase Corpus](https://mlsanity.com/paper/2103.13103/) (2021)
- [Graphine: A Dataset for Graph-aware Terminology Definition Generation](https://mlsanity.com/paper/2109.04018/) (2021)
- [Improving Large-scale Paraphrase Acquisition and Generation](https://mlsanity.com/paper/2210.03235/) (2022)
- [Generative Pretraining for Paraphrase Evaluation](https://mlsanity.com/paper/2107.08251/) (2021)
- [Evaluation of pretrained language models on music understanding](https://mlsanity.com/paper/2409.11449/) (2024)
- [Fake News Detection After LLM Laundering: Measurement and Explanation](https://mlsanity.com/paper/2501.18649/) (2025)
- [Evade the Trap of Mediocrity: Promoting Diversity and Novelty in Text Generation via Concentrating Attention](https://mlsanity.com/paper/2211.07164/) (2022)
- [Active Learning for Natural Language Generation](https://mlsanity.com/paper/2305.15040/) (2023)
- [ARC-Encoder: learning compressed text representations for large language models](https://mlsanity.com/paper/2510.20535/) (2025)
- [Beyond LLM-as-a-Judge: Deterministic Metrics for Multilingual Generative Text Evaluation](https://mlsanity.com/paper/2604.05083/) (2026)
- [What's Mine becomes Yours: Defining, Annotating and Detecting Context-Dependent Paraphrases in News Interview Dialogs](https://mlsanity.com/paper/2404.06670/) (2024)
- [A Large-Scale Benchmark for Vietnamese Sentence Paraphrases](https://mlsanity.com/paper/2502.07188/) (2025)
- [What Makes a Good Paraphrase: Do Automated Evaluations Work?](https://mlsanity.com/paper/2307.14818/) (2023)
- [G-SPEED: General SParse Efficient Editing MoDel](https://mlsanity.com/paper/2310.10480/) (2023)
- [The power of text similarity in identifying AI-LLM paraphrased documents: The case of BBC news articles and ChatGPT](https://mlsanity.com/paper/2505.12405/) (2025)
- [Hierarchical Wasserstein Merging for Multi-Domain Multi-Task Learning: From Specialists to a Generalist](https://mlsanity.com/paper/2609.06406/) (2026)

## Abstract (arXiv)

We propose ParaSCI, the first large-scale paraphrase dataset in the scientific field, including 33,981 paraphrase pairs from ACL (ParaSCI-ACL) and 316,063 pairs from arXiv (ParaSCI-arXiv). Digging into characteristics and common patterns of scientific papers, we construct this dataset though intra-paper and inter-paper methods, such as collecting citations to the same paper or aggregating definitions by scientific terms. To take advantage of sentences paraphrased partially, we put up PDBERT as a general paraphrase discovering method. The major advantages of paraphrases in ParaSCI lie in the prominent length and textual diversity, which is complementary to existing paraphrase datasets. ParaSCI obtains satisfactory results on human evaluation and downstream tasks, especially long paraphrase generation.