# PeerArg: Argumentative Peer Review with LLMs

arXiv: https://arxiv.org/abs/2409.16813 · HTML: https://mlsanity.com/paper/2409.16813/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 23 references that are in the map.
- First posted on arXiv 25 Sep 2024.

## Builds on (most cited first, 9 of 9)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications](https://mlsanity.com/paper/1804.09635/) (2018)
- [Argument Mining for Understanding Peer Reviews](https://mlsanity.com/paper/1903.10104/) (2019)
- [Interpreting Neural Networks as Gradual Argumentation Frameworks (Including Proof Appendix)](https://mlsanity.com/paper/2012.05738/) (2020)
- [MOPRD: A multidisciplinary open peer review dataset](https://mlsanity.com/paper/2212.04972/) (2022)
- [Aspect-based Sentiment Analysis of Scientific Reviews](https://mlsanity.com/paper/2006.03257/) (2020)
- [Modular Semantics and Characteristics for Bipolar Weighted Argumentation Graphs](https://mlsanity.com/paper/1807.06685/) (2018)
- [Can Large Language Models perform Relation-based Argument Mining?](https://mlsanity.com/paper/2402.11243/) (2024)
- [Argument Mining Driven Analysis of Peer-Reviews](https://mlsanity.com/paper/2012.07743/) (2020)

## Built on by (most cited first, 9 of 9)

- [Is Your Paper Being Reviewed by an LLM? Benchmarking AI Text Detection in Peer Review](https://mlsanity.com/paper/2502.19614/) (2025)
- [From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systems](https://mlsanity.com/paper/2503.01424/) (2025)
- [Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future](https://mlsanity.com/paper/2604.27924/) (2026)
- ["Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers](https://mlsanity.com/paper/2511.01287/) (2025)
- [Set Contribution Functions for Quantitative Bipolar Argumentation and their Principles](https://mlsanity.com/paper/2509.14963/) (2025)
- [Argumentative Human-AI Decision-Making: Toward AI Agents That Reason With Us, Not For Us](https://mlsanity.com/paper/2603.15946/) (2026)
- [LLM-based Argument Mining meets Argumentation and Description Logics: a Unified Framework for Reasoning about Debates](https://mlsanity.com/paper/2603.02858/) (2026)
- [Beyond Detection: Governing GenAI in Academic Peer Review as a Sociotechnical Challenge](https://mlsanity.com/paper/2603.20214/) (2026)
- [Who Argues What? Joint Argument-Entity Detection and Classification in Political Debates](https://mlsanity.com/paper/2609.10192/) (2026)

## Abstract (arXiv)

Peer review is an essential process to determine the quality of papers submitted to scientific conferences or journals. However, it is subjective and prone to biases. Several studies have been conducted to apply techniques from NLP to support peer review, but they are based on black-box techniques and their outputs are difficult to interpret and trust. In this paper, we propose a novel pipeline to support and understand the reviewing and decision-making processes of peer review: the PeerArg system combining LLMs with methods from knowledge representation. PeerArg takes in input a set of reviews for a paper and outputs the paper acceptance prediction. We evaluate the performance of the PeerArg pipeline on three different datasets, in comparison with a novel end-2-end LLM that uses few-shot learning to predict paper acceptance given reviews. The results indicate that the end-2-end LLM is capable of predicting paper acceptance from reviews, but a variant of the PeerArg pipeline outperforms this LLM.