# PARADE: A New Dataset for Paraphrase Identification Requiring Computer Science Domain Knowledge

arXiv: https://arxiv.org/abs/2010.03725 · HTML: https://mlsanity.com/paper/2010.03725/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2020.

## Built on by (most cited first, 7 of 7)

- [CSFCube -- A Test Collection of Computer Science Research Articles for Faceted Query by Example](https://mlsanity.com/paper/2103.12906/) (2021)
- [Finnish Paraphrase Corpus](https://mlsanity.com/paper/2103.13103/) (2021)
- [Match-Prompt: Improving Multi-task Generalization Ability for Neural Text Matching via Prompt Learning](https://mlsanity.com/paper/2204.02725/) (2022)
- [Finding a Needle in the Adversarial Haystack: A Targeted Paraphrasing Approach For Uncovering Edge Cases with Minimal Distribution Distortion](https://mlsanity.com/paper/2401.11373/) (2024)
- [Semantic Search as Extractive Paraphrase Span Detection](https://mlsanity.com/paper/2112.04886/) (2021)
- [Closing the Operational Gap in Semantic Caching](https://mlsanity.com/paper/2606.19719/) (2026)
- [Retrieve, Generate, Evaluate: A Case Study for Medical Paraphrases Generation with Small Language Models](https://mlsanity.com/paper/2407.16565/) (2024)

## Abstract (arXiv)

We present a new benchmark dataset called PARADE for paraphrase identification that requires specialized domain knowledge. PARADE contains paraphrases that overlap very little at the lexical and syntactic level but are semantically equivalent based on computer science domain knowledge, as well as non-paraphrases that overlap greatly at the lexical and syntactic level but are not semantically equivalent based on this domain knowledge. Experiments show that both state-of-the-art neural models and non-expert human annotators have poor performance on PARADE. For example, BERT after fine-tuning achieves an F1 score of 0.709, which is much lower than its performance on other paraphrase identification datasets. PARADE can serve as a resource for researchers interested in testing models that incorporate domain knowledge. We make our data and code freely available.