# ReviewRobot: Explainable Paper Review Generation based on Knowledge Synthesis

arXiv: https://arxiv.org/abs/2010.06119 · HTML: https://mlsanity.com/paper/2010.06119/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2020.

## Built on by (most cited first, 30 of 54)

- [ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing](https://mlsanity.com/paper/2306.00622/) (2023)
- [MARG: Multi-Agent Review Generation for Scientific Papers](https://mlsanity.com/paper/2401.04259/) (2024)
- [DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Process](https://mlsanity.com/paper/2503.08569/) (2025)
- [OpenReviewer: A Specialized Large Language Model for Generating Critical Scientific Paper Reviews](https://mlsanity.com/paper/2412.11948/) (2024)
- [Can We Automate Scientific Reviewing?](https://mlsanity.com/paper/2102.00176/) (2021)
- [LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing](https://mlsanity.com/paper/2406.16253/) (2024)
- [Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation](https://mlsanity.com/paper/2502.05151/) (2025)
- [Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review](https://mlsanity.com/paper/2204.10805/) (2022)
- [Automated Focused Feedback Generation for Scientific Writing Assistance](https://mlsanity.com/paper/2405.20477/) (2024)
- [Automated Peer Reviewing in Paper SEA: Standardization, Evaluation, and Analysis](https://mlsanity.com/paper/2407.12857/) (2024)
- [Investigating Fairness Disparities in Peer Review: A Language Model Enhanced Approach](https://mlsanity.com/paper/2211.06398/) (2022)
- [Automated scholarly paper review: Concepts, technologies, and challenges](https://mlsanity.com/paper/2111.07533/) (2021)
- [Human Evaluation of Creative NLG Systems: An Interdisciplinary Survey on Recent Papers](https://mlsanity.com/paper/2108.00308/) (2021)
- [A Benchmark Corpus for the Detection of Automatically Generated Text in Academic Publications](https://mlsanity.com/paper/2202.02013/) (2022)
- [The Good, the Bad and the Constructive: Automatically Measuring Peer Review's Utility for Authors](https://mlsanity.com/paper/2509.04484/) (2025)
- [How Far Are AI Scientists from Changing the World?](https://mlsanity.com/paper/2507.23276/) (2025)
- [Scientific Opinion Summarization: Paper Meta-review Generation Dataset, Methods, and Evaluation](https://mlsanity.com/paper/2305.14647/) (2023)
- [Controllable and Diverse Text Generation in E-commerce](https://mlsanity.com/paper/2102.11497/) (2021)
- [Unveiling the Merits and Defects of LLMs in Automatic Review Generation for Scientific Papers](https://mlsanity.com/paper/2509.19326/) (2025)
- [REMOR: Automated Peer Review Generation with LLM Reasoning and Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2505.11718/) (2025)
- [Insights from the ICLR Peer Review and Rebuttal Process](https://mlsanity.com/paper/2511.15462/) (2025)
- [SmartBook: AI-Assisted Situation Report Generation for Intelligence Analysts](https://mlsanity.com/paper/2303.14337/) (2023)
- [Leveraging Professional Radiologists' Expertise to Enhance LLMs' Evaluation for Radiology Reports](https://mlsanity.com/paper/2401.16578/) (2024)
- [NovBench: Evaluating Large Language Models on Academic Paper Novelty Assessment](https://mlsanity.com/paper/2604.11543/) (2026)
- [Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future](https://mlsanity.com/paper/2604.27924/) (2026)
- [A Multi-Task Evaluation of LLMs' Processing of Academic Text Input](https://mlsanity.com/paper/2508.11779/) (2025)
- [Determining the Credibility of Science Communication](https://mlsanity.com/paper/2105.14473/) (2021)
- [Did AI get more negative recently?](https://mlsanity.com/paper/2202.13610/) (2022)
- [Identifying Aspects in Peer Reviews](https://mlsanity.com/paper/2504.06910/) (2025)
- [A Human-AI Collaborative Workflow for Mathematical Discovery: A Case Study in Grover-Compatible Riemannian Optimization](https://mlsanity.com/paper/2512.09443/) (2025)

## Abstract (arXiv)

To assist human review process, we build a novel ReviewRobot to automatically assign a review score and write comments for multiple categories such as novelty and meaningful comparison. A good review needs to be knowledgeable, namely that the comments should be constructive and informative to help improve the paper; and explainable by providing detailed evidence. ReviewRobot achieves these goals via three steps: (1) We perform domain-specific Information Extraction to construct a knowledge graph (KG) from the target paper under review, a related work KG from the papers cited by the target paper, and a background KG from a large collection of previous papers in the domain. (2) By comparing these three KGs, we predict a review score and detailed structured knowledge as evidence for each review category. (3) We carefully select and generalize human review sentences into templates, and apply these templates to transform the review scores and evidence into natural language comments. Experimental results show that our review score predictor reaches 71.4%-100% accuracy. Human assessment by domain experts shows that 41.7%-70.5% of the comments generated by ReviewRobot are valid and constructive, and better than human-written ones for 20% of the time. Thus, ReviewRobot can serve as an assistant for paper reviewers, program chairs and authors.