mlsanityData to 7 Oct 2026

A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications

arXiv 1804.09635 ↗First posted 25 Apr 2018cs.CL cs.AIcited by 135

Not on the map. Why?

Abstract (arXiv)

Peer reviewing is a central component in the scientific publishing process. We present the first public dataset of scientific peer reviews available for research purposes (PeerRead v1) providing an opportunity to study this important artifact. The dataset consists of 14.7K paper drafts and the corresponding accept/reject decisions in top-tier venues including ACL, NIPS and ICLR. The dataset also includes 10.7K textual peer reviews written by experts for a subset of the papers. We describe the data collection process and report interesting observed phenomena in the peer reviews. We also propose two novel NLP tasks based on this dataset and provide simple baseline models. In the first task, we show that simple models can predict whether a paper is accepted with up to 21% error reduction compared to the majority baseline. In the second task, we predict the numerical scores of review aspects and show that simple models can outperform the mean baseline for aspects with high variance such as 'originality' and 'impact'.

Key facts. Cited by 135 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 15; 2023: 8; 2024: 24; 2025: 34; 2026: 44.

135 papers.

Sort:

By area:

  1. 2022Self-critiquing models for assisting human evaluatorscited by 372
  2. 2023M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detectioncited by 118
  3. 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 118
  4. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 101
  5. 2025DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Processcited by 97
  6. 2022CREPE: Open-Domain Question Answering with False Presuppositionscited by 70
  7. 2021Can We Automate Scientific Reviewing?cited by 63
  8. 2024OpenReviewer: A Specialized Large Language Model for Generating Critical Scientific Paper Reviewscited by 63
  9. 2024LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewingcited by 62
  10. 2022NLPeer: A Unified Resource for the Computational Study of Peer Reviewcited by 60
  11. 2021Identifiable Deep Generative Models via Sparse Decodingcited by 54
  12. 2024DeTeCtive: Detecting AI-generated Text via Multi-Level Contrastive Learningcited by 52
38 more
  1. 2024CiteME: Can Language Models Accurately Cite Scientific Claims?cited by 45
  2. 2021MReD: A Meta-Review Dataset for Structure-Controllable Text Generationcited by 42
  3. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 41
  4. 2022Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Reviewcited by 37
  5. 2024Unveiling the Generalization Power of Fine-Tuned Large Language Modelscited by 35
  6. 2024Automated Focused Feedback Generation for Scientific Writing Assistancecited by 33
  7. 2024Automated Peer Reviewing in Paper SEA: Standardization, Evaluation, and Analysiscited by 33
  8. 2022MOPRD: A multidisciplinary open peer review datasetcited by 32
  9. 2025TreeReview: A Dynamic Tree of Questions Framework for Deep and Efficient LLM-based Scientific Peer Reviewcited by 31
  10. 2025ReviewAgents: Bridging the Gap Between Human and AI-Generated Paper Reviewscited by 28
  11. 2021CSDS: A Fine-Grained Chinese Dataset for Customer Service Dialogue Summarizationcited by 27
  12. 2022arXivEdits: Understanding the Human Revision Process in Scientific Writingcited by 26
  13. 2024What Can Natural Language Processing Do for Peer Review?cited by 24
  14. 2022Investigating Fairness Disparities in Peer Review: A Language Model Enhanced Approachcited by 23
  15. 2025ReviewEval: An Evaluation Framework for AI-Generated Reviewscited by 22
  16. 2021Automated scholarly paper review: Concepts, technologies, and challengescited by 21
  17. 2025PeerQA: A Scientific Question Answering Dataset from Peer Reviewscited by 21
  18. 2023Automatic Analysis of Substantiation in Scientific Peer Reviewscited by 19
  19. 2025Re2: A Consistency-ensured Dataset for Full-stage Peer Review and Multi-turn Rebuttal Discussionscited by 19
  20. 2023Exploring Jiu-Jitsu Argumentation for Writing Peer Review Rebuttalscited by 18
  21. 2025Detecting LLM-Generated Peer Reviewscited by 18
  22. 2025All That Glitters is Not Novel: Plagiarism in AI Generated Researchcited by 17
  23. 2025ReviewRL: Towards Automated Scientific Review with RLcited by 17
  24. 2024SciDQA: A Deep Reading Comprehension Dataset over Scientific Paperscited by 15
  25. 2025How Far Are AI Scientists from Changing the World?cited by 15
  26. 2024CASIMIR: A Corpus of Scientific Articles enhanced with Multiple Author-Integrated Revisionscited by 14
  27. 2025Breaking the Reviewer: Assessing the Vulnerability of Large Language Models in Automated Peer Review Under Textual Adversarial Attackscited by 14
  28. 2022Yes-Yes-Yes: Proactive Data Collection for ACL Rolling Review and Beyondcited by 12
  29. 2024Evaluating the Predictive Capacity of ChatGPT for Academic Peer Review Outcomes Across Multiple Platformscited by 11
  30. 2024A Practical Examination of AI-Generated Text Detectors for Large Language Modelscited by 11
  31. 2022SentBS: Sentence-level Beam Search for Controllable Summarizationcited by 10
  32. 2025Help Me Write a Story: Evaluating LLMs' Ability to Generate Writing Feedbackcited by 10
  33. 2025Paper Copilot: Tracking the Evolution of Peer Review in AI Conferencescited by 10
  34. 2025ScholarEval: Research Idea Evaluation Grounded in Literaturecited by 10
  35. 2024PeerArg: Argumentative Peer Review with LLMscited by 9
  36. 2025REMOR: Automated Peer Review Generation with LLM Reasoning and Multi-Objective Reinforcement Learningcited by 9
  37. 2025MMReview: A Multidisciplinary and Multimodal Benchmark for LLM-Based Peer Review Automationcited by 9
  38. 2026RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Modelscited by 9

50 most cited of 135 All 135 (JSON)