mlsanityData to 7 Oct 2026

COMPARE: A Taxonomy and Dataset of Comparison Discussions in Peer Reviews

arXiv 2108.04366 ↗First posted 9 Aug 2021cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Comparing research papers is a conventional method to demonstrate progress in experimental research. We present COMPARE, a taxonomy and a dataset of comparison discussions in peer reviews of research papers in the domain of experimental deep learning. From a thorough observation of a large set of review sentences, we build a taxonomy of categories in comparison discussions and present a detailed annotation scheme to analyze this. Overall, we annotate 117 reviews covering 1,800 sentences. We experiment with various methods to identify comparison sentences in peer reviews and report a maximum F1 Score of 0.49. We also pretrain two language models specifically on ML, NLP, and CV paper abstracts and reviews to learn informative representations of peer reviews. The annotated dataset and the pretrained models are available at https://github.com/shruti-singh/COMPARE .

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 10 of its 17 references that are in the map. First posted on arXiv 9 Aug 2021.

Builds on

10 of 17 references.

References · 10

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  4. 2018Universal Sentence Encodercited by 485
  5. 2018A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applicationscited by 135
  6. 2020SciREX: A Challenge Dataset for Document-Level Information Extractioncited by 111
  7. 2019Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Constructioncited by 50
  8. 2019Argument Mining for Understanding Peer Reviewscited by 41
  9. 2020Aspect-based Sentiment Analysis of Scientific Reviewscited by 24
  10. 2020Argument Mining Driven Analysis of Peer-Reviewscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 0; 2024: 0; 2025: 1; 2026: 1.

4 papers.

  1. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 41
  2. 2022MOPRD: A multidisciplinary open peer review datasetcited by 32
  3. 2021Automated scholarly paper review: Concepts, technologies, and challengescited by 21
  4. 2026APRES: An Agentic Paper Revision and Evaluation Systemcited by 1