mlsanityData to 7 Oct 2026

MARG: Multi-Agent Review Generation for Scientific Papers

arXiv 2401.04259 ↗First posted 8 Jan 2024cs.CLcited by 98

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

We study the ability of LLMs to generate feedback for scientific papers and develop MARG, a feedback generation approach using multiple LLM instances that engage in internal discussion. By distributing paper text across agents, MARG can consume the full text of papers beyond the input length limitations of the base LLM, and by specializing agents and incorporating sub-tasks tailored to different comment types (experiments, clarity, impact) it improves the helpfulness and specificity of feedback. In a user study, baseline methods using GPT-4 were rated as producing generic or very generic comments more than half the time, and only 1.7 comments per paper were rated as good overall in the best baseline. Our system substantially improves the ability of GPT-4 to generate specific and helpful feedback, reducing the rate of generic comments from 60% to 29% and generating 3.7 good comments per paper (a 2.2x improvement).

Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 20 of its 41 references that are in the map. First posted on arXiv 8 Jan 2024.

Builds on

20 of 41 references.

References · 20

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  4. 2020Longformer: The Long-Document Transformercited by 3,362
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  6. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  7. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  8. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  9. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  10. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  11. 2021Recursively Summarizing Books with Human Feedbackcited by 287
  12. 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaborationcited by 246
8 more
  1. 2023Can large language models provide useful feedback on research papers? A large-scale empirical analysiscited by 195
  2. 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 118
  3. 2023Retrieval meets Long Context Large Language Modelscited by 114
  4. 2023Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Modelscited by 62
  5. 2020ReviewRobot: Explainable Paper Review Generation based on Knowledge Synthesiscited by 53
  6. 2023Writer-Defined AI Personas for On-Demand Feedback Generationcited by 36
  7. 2023ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviewscited by 34
  8. 2023Writing Tools: Looking Back to Look Aheadcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 47; 2026: 37.

98 papers.

Sort:

By area:

  1. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  2. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
  3. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 348
  4. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 221
  5. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  6. 2025DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Processcited by 97
  7. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  8. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
  9. 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 67
  10. 2025AgentRxiv: Towards Collaborative Autonomous Researchcited by 64
  11. 2025Large language models for automated scholarly paper review: A surveycited by 59
  12. 2024LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptionscited by 45
38 more
  1. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 41
  2. 2025Automated Hypothesis Validation with Agentic Sequential Falsificationscited by 40
  3. 2025TreeReview: A Dynamic Tree of Questions Framework for Deep and Efficient LLM-based Scientific Peer Reviewcited by 31
  4. 2024CHIME: LLM-Assisted Hierarchical Organization of Scientific Studies for Literature Review Supportcited by 28
  5. 2025ReviewAgents: Bridging the Gap Between Human and AI-Generated Paper Reviewscited by 28
  6. 2024AgentCoord: Visually Exploring Coordination Strategy for LLM-based Multi-Agent Collaborationcited by 26
  7. 2024What Can Natural Language Processing Do for Peer Review?cited by 24
  8. 2025The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systemscited by 24
  9. 2025Mind the Blind Spots: A Focus-Level Evaluation Framework for LLM Reviewscited by 23
  10. 2025Creativity in LLM-based Multi-Agent Systems: A Surveycited by 23
  11. 2025PeerQA: A Scientific Question Answering Dataset from Peer Reviewscited by 21
  12. 2025Is Your Paper Being Reviewed by an LLM? Benchmarking AI Text Detection in Peer Reviewcited by 18
  13. 2025Detecting LLM-Generated Peer Reviewscited by 18
  14. 2024AIGS: Generating Science from AI-Powered Automated Falsificationcited by 17
  15. 2025The Good, the Bad and the Constructive: Automatically Measuring Peer Review's Utility for Authorscited by 16
  16. 2024Usefulness of LLMs as an Author Checklist Assistant for Scientific Papers: NeurIPS'24 Experimentcited by 15
  17. 2025How Far Are AI Scientists from Changing the World?cited by 15
  18. 2025OmniScientist: Toward a Co-evolving Ecosystem of Human and AI Scientistscited by 15
  19. 2025MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detectioncited by 14
  20. 2025Reviewing Scientific Papers for Critical Problems With Reasoning LLMs: Baseline Approaches and Automatic Evaluationcited by 12
  21. 2025AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Researchcited by 12
  22. 2025Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedbackcited by 12
  23. 2025Unveiling the Merits and Defects of LLMs in Automatic Review Generation for Scientific Paperscited by 12
  24. 2025FLAWS: A Benchmark for Error Identification and Localization in Scientific Paperscited by 12
  25. 2025From Replication to Redesign: Exploring Pairwise Comparisons for LLM-Based Peer Reviewcited by 11
  26. 2025Help Me Write a Story: Evaluating LLMs' Ability to Generate Writing Feedbackcited by 10
  27. 2025Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Frameworkcited by 10
  28. 2025Prompt Injection Attacks on LLM Generated Reviews of Scientific Publicationscited by 10
  29. 2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMscited by 10
  30. 2025REMOR: Automated Peer Review Generation with LLM Reasoning and Multi-Objective Reinforcement Learningcited by 9
  31. 2025MMReview: A Multidisciplinary and Multimodal Benchmark for LLM-Based Peer Review Automationcited by 9
  32. 2025From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systemscited by 8
  33. 2025Automatic Evaluation Metrics for Artificially Generated Scientific Researchcited by 8
  34. 2025Position: The ML Community Must Build an AI-Augmented Peer-Review Ecosystemcited by 8
  35. 2026Paper2Rebuttal: A Multi-Agent Framework for Transparent Author Response Assistancecited by 8
  36. 2026AI Can Learn Scientific Tastecited by 8
  37. 2025Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Modelscited by 7
  38. 2025XtraGPT: Context-Aware and Controllable Academic Paper Revision via Human-AI Collaborationcited by 6

50 most cited of 98 All 98 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026From Passive Generation to Investigation: A Proactive Scientific Peer Review Agent
  2. 2025FRAME: Feedback-Refined Agent Methodology for Enhancing Medical Research Insights
  3. 2024ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models
  4. 2026SurveyAgent-HKA: A multi-agent framework for scientific survey generation with LLMs and human knowledge augmentation
  5. 2025From First Draft to Final Insight: A Multi-Agent Approach for Feedback Generation
  6. 2025LiRA: A Multi-Agent Framework for Reliable and Readable Literature Review Generation
  7. 2026GoodPoint: Learning Constructive Scientific Paper Feedback from Author Responses
  8. 2025ReviewRL: Towards Automated Scientific Review with RL