mlsanityData to 14 Oct 2026

Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future

arXiv 2604.27924 ↗First posted 30 Apr 2026cs.CL cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Peer review is a multi-stage process involving reviews, rebuttals, meta-reviews, final decisions, and subsequent manuscript revisions. Recent advances in large language models (LLMs) have motivated methods that assist or automate different stages of this pipeline. In this survey, we synthesize techniques for (i) peer review generation, including fine-tuning strategies, agent-based systems, RL-based methods, and emerging paradigms to enhance generation; (ii) after-review tasks including rebuttals, meta-review and revision aligned to reviews; and (iii) evaluation methods spanning human-centered, reference-based, LLM-based and aspect-oriented. We catalog datasets, compare modeling choices, and discuss limitations, ethical concerns, and future directions. The survey aims to provide practical guidance for building, evaluating, and integrating LLM systems across the full peer review workflow.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 95 of its 155 references that are in the map. First posted on arXiv 30 Apr 2026.

Builds on

95 of 155 references.

References · 95

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  7. 2017Deep reinforcement learning from human preferencescited by 3,912
  8. 2023A Survey of Large Language Modelscited by 2,576
  9. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  10. 2024A Survey on LLM-as-a-Judgecited by 1,423
  11. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  12. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
38 more
  1. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  2. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  3. 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 431
  4. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
  5. 2025Chain of Draft: Thinking Faster by Writing Lesscited by 297
  6. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 267
  7. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  8. 2023Can large language models provide useful feedback on research papers? A large-scale empirical analysiscited by 200
  9. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 170
  10. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 164
  11. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 148
  12. 2018A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applicationscited by 136
  13. 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 122
  14. 2023The Semantic Scholar Open Data Platformcited by 118
  15. 2021Machine Learning with a Reject Option: A surveycited by 109
  16. 2025DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Processcited by 100
  17. 2024AgentReview: Exploring Peer Review Dynamics with LLM Agentscited by 93
  18. 2020RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Realcited by 91
  19. 2018ROUGE 2.0: Updated and Improved Measures for Evaluation of Summarization Taskscited by 67
  20. 2024OpenReviewer: A Specialized Large Language Model for Generating Critical Scientific Paper Reviewscited by 65
  21. 2021Can We Automate Scientific Reviewing?cited by 63
  22. 2024LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewingcited by 62
  23. 2025Large language models for automated scholarly paper review: A surveycited by 61
  24. 2025All Roads Lead to Likelihood: The Value of Reinforcement Learning in Fine-Tuningcited by 59
  25. 2020ReviewRobot: Explainable Paper Review Generation based on Knowledge Synthesiscited by 54
  26. 2023A Practical Survey on Zero-shot Prompt Design for In-context Learningcited by 49
  27. 2024Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactionscited by 46
  28. 2025RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environmentscited by 46
  29. 2024AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviewscited by 44
  30. 2021MReD: A Meta-Review Dataset for Structure-Controllable Text Generationcited by 42
  31. 2023GPT4 is Slightly Helpful for Peer-Review Assistance: A Pilot Studycited by 40
  32. 2021DISAPERE: A Dataset for Discourse Structure in Peer Review Discussionscited by 36
  33. 2023ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviewscited by 34
  34. 2025LettuceDetect: A Hallucination Detection Framework for RAG Applicationscited by 34
  35. 2024Automated Focused Feedback Generation for Scientific Writing Assistancecited by 33
  36. 2022MOPRD: A multidisciplinary open peer review datasetcited by 32
  37. 2025ReviewAgents: Bridging the Gap Between Human and AI-Generated Paper Reviewscited by 32
  38. 2019Does My Rebuttal Matter? Insights from a Major NLP Conferencecited by 31

50 most cited of 95 All 95 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026No Hidden Prompts Needed! You Can Game AI Peer Review with Presentation-Only Revisionscited by 2
  2. 2026Review Arcade: On the Human Alignment and Gameability of LLM Reviewscited by 1
  3. 2026AI-Assisted Peer Review Across Research Communities: From Reviewer AI Policies to LLM Review Qualitycited by 1
  4. 2026The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishingcited by 1
  5. 2026ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generationcited by 0
  6. 2026NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessmentcited by 0