mlsanityData to 14 Oct 2026

ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews

arXiv 2306.12587 ↗First posted 21 Jun 2023cs.CLcited by 34

Not on the map. Why?

Abstract (arXiv)

We introduce the task of automatically revising scientific papers based on peer feedback and release ARIES, a dataset of review comments and their corresponding paper edits. The data is drawn from real reviewer-author interactions from computer science, and we provide labels linking each reviewer comment to the specific paper edits made by the author in response. We automatically create a high-precision silver training set, as well as an expert-labeled test set that shows high inter-annotator agreement. In experiments with 10 models covering the state of the art, we find that they struggle even to identify which edits correspond to a comment -- especially when the relationship between the edit and the comment is indirect and requires reasoning to uncover. We also extensively analyze GPT-4's ability to generate edits given a comment and the original paper. We find that it often succeeds on a superficial level, but tends to rigidly follow the wording of the feedback rather than the underlying intent, and lacks technical details compared to human-written edits.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 34 of its 50 references that are in the map. First posted on arXiv 21 Jun 2023.

Builds on

34 of 50 references.

References · 34

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  4. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  5. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  6. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 249
  7. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 211
  8. 2022SciRepEval: A Multi-Format Benchmark for Scientific Document Representationscited by 129
  9. 2021A Recipe For Arbitrary Text Style Transfer with Large Language Modelscited by 107
  10. 2020Neural CRF Model for Sentence Alignment in Text Simplificationcited by 106
  11. 2019Encode, Tag, Realize: High-Precision Text Editingcited by 99
  12. 2022PEER: A Collaborative Language Modelcited by 86
22 more
  1. 2023CoEdIT: Text Editing by Task-Specific Instruction Tuningcited by 68
  2. 2022Understanding Iterative Revision from Human-Written Textcited by 57
  3. 2022Read, Revise, Repeat: A System Demonstration for Human-in-the-loop Iterative Text Revisioncited by 46
  4. 2022On Improving Summarization Factual Consistency from Natural Language Feedbackcited by 38
  5. 2022Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Reviewcited by 37
  6. 2020Explaining Relationships Between Scientific Documentscited by 35
  7. 2022EdiT5: Semi-Autoregressive Text-Editing with T5 Warm-Startcited by 35
  8. 2020Text Editing by Commandcited by 33
  9. 2022Learning to Model Editing Processescited by 30
  10. 2022arXivEdits: Understanding the Human Revision Process in Scientific Writingcited by 26
  11. 2022A Computational Inflection for Scientific Discoverycited by 23
  12. 2022Augmenting Scientific Creativity with an Analogical Search Enginecited by 22
  13. 2023CiteSee: Augmenting Citations in Scientific Papers with Persistent and Personalized Historical Contextcited by 22
  14. 2022Towards Automated Document Revision: Grammatical Error Correction, Fluency Edits, and Beyondcited by 20
  15. 2022Improving Iterative Text Revision by Learning Where to Edit from Other Revision Taskscited by 20
  16. 2023Beyond Summarization: Designing AI Support for Real-World Expository Writing Taskscited by 20
  17. 2021Relating Neural Text Degeneration to Exposure Biascited by 19
  18. 2022CORWA: A Citation-Oriented Related Work Annotation Datasetcited by 17
  19. 2020Langsmith: An Interactive Academic Text Revision Systemcited by 13
  20. 2020Conversations with Documents. An Exploration of Document-Centered Assistancecited by 12
  21. 2022ArgRewrite V.2: an Annotated Argumentative Revisions Corpuscited by 9
  22. 2023Writing Tools: Looking Back to Look Aheadcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 13; 2026: 11.

34 papers.

Sort:

By area:

  1. 2024MARG: Multi-Agent Review Generation for Scientific Paperscited by 104
  2. 2024A Design Space for Intelligent and Interactive Writing Assistantscited by 76
  3. 2024Aligning LLM Agents by Learning Latent Preference from User Editscited by 69
  4. 2025Large language models for automated scholarly paper review: A surveycited by 61
  5. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 42
  6. 2024What Can Natural Language Processing Do for Peer Review?cited by 25
  7. 2025Re2: A Consistency-ensured Dataset for Full-stage Peer Review and Multi-turn Rebuttal Discussionscited by 23
  8. 2025PeerQA: A Scientific Question Answering Dataset from Peer Reviewscited by 22
  9. 2024Usefulness of LLMs as an Author Checklist Assistant for Scientific Papers: NeurIPS'24 Experimentcited by 16
  10. 2024SciDQA: A Deep Reading Comprehension Dataset over Scientific Paperscited by 15
  11. 2025How Far Are AI Scientists from Changing the World?cited by 15
  12. 2024CASIMIR: A Corpus of Scientific Articles enhanced with Multiple Author-Integrated Revisionscited by 14
22 more
  1. 2025CLAIMCHECK: How Grounded are LLM Critiques of Scientific Papers?cited by 12
  2. 2024Re3: A Holistic Framework and Dataset for Modeling Collaborative Document Revisioncited by 11
  3. 2025ScholaWrite: A Dataset of End-to-End Scholarly Writing Processcited by 9
  4. 2024'Quis custodiet ipsos custodes?' Who will watch the watchmen? On Detecting AI-generated peer-reviewscited by 8
  5. 2025eRevise+RF: A Writing Evaluation System for Assessing Student Essay Revisions and Providing Formative Feedbackcited by 8
  6. 2025From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systemscited by 8
  7. 2024Acceleron: A Tool to Accelerate Research Ideationcited by 6
  8. 2025Large Language Models Penetration in Scholarly Writing and Peer Reviewcited by 6
  9. 2026Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Futurecited by 6
  10. 2026Author-in-the-Loop Response Generation and Evaluation: Integrating Author Expertise and Intent in Responses to Peer Reviewcited by 4
  11. 2026Defend: Automated Rebuttals for Peer Review with Minimal Author Guidancecited by 4
  12. 2025Efficient Layer-wise LLM Fine-tuning for Revision Intention Predictioncited by 3
  13. 2026Expos'ia: Teaching and Assessment of Academic Writing Skills for Research Project Proposals and Peer Feedbackcited by 1
  14. 2025ABCD-LINK: Annotation Bootstrapping for Cross-Document Fine-Grained Linkscited by 0
  15. 2025ResearchArcade: Graph Interface for Academic Taskscited by 0
  16. 2026Principled Fine-tuning of LLMs from User-Edits: A Medley of Preference, Supervision, and Rewardcited by 0
  17. 2026Intention-Adaptive LLM Fine-Tuning for Text Revision Generationcited by 0
  18. 2026Feedback-to-Rubrics: Can We Learn Expert Criteria from Inline Comments?cited by 0
  19. 2026Heterogeneous Element-Aware Cross-Version Differencing of Scientific Documents via Layout-Aware Alignment and Structure-Aware Reasoningcited by 0
  20. 2026Metag: A dataset to build agentic meta-reviewing capabilitiescited by 0
  21. 2026Control-Data Flow Separation: Stable Prompt Optimization in Multi-Agent LLMscited by 0
  22. 2026ReGround: Grounding Reviewer Comments in Multimodal Evidencecited by 0