# ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews

arXiv: https://arxiv.org/abs/2306.12587 · HTML: https://mlsanity.com/paper/2306.12587/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 50 references that are in the map.
- First posted on arXiv 21 Jun 2023.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [LinkBERT: Pretraining Language Models with Document Links](https://mlsanity.com/paper/2203.15827/) (2022)
- [SciRepEval: A Multi-Format Benchmark for Scientific Document Representations](https://mlsanity.com/paper/2211.13308/) (2022)
- [A Recipe For Arbitrary Text Style Transfer with Large Language Models](https://mlsanity.com/paper/2109.03910/) (2021)
- [Neural CRF Model for Sentence Alignment in Text Simplification](https://mlsanity.com/paper/2005.02324/) (2020)
- [Encode, Tag, Realize: High-Precision Text Editing](https://mlsanity.com/paper/1909.01187/) (2019)
- [PEER: A Collaborative Language Model](https://mlsanity.com/paper/2208.11663/) (2022)
- [CoEdIT: Text Editing by Task-Specific Instruction Tuning](https://mlsanity.com/paper/2305.09857/) (2023)
- [Understanding Iterative Revision from Human-Written Text](https://mlsanity.com/paper/2203.03802/) (2022)
- [Read, Revise, Repeat: A System Demonstration for Human-in-the-loop Iterative Text Revision](https://mlsanity.com/paper/2204.03685/) (2022)
- [On Improving Summarization Factual Consistency from Natural Language Feedback](https://mlsanity.com/paper/2212.09968/) (2022)
- [Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review](https://mlsanity.com/paper/2204.10805/) (2022)
- [Explaining Relationships Between Scientific Documents](https://mlsanity.com/paper/2002.00317/) (2020)
- [EdiT5: Semi-Autoregressive Text-Editing with T5 Warm-Start](https://mlsanity.com/paper/2205.12209/) (2022)
- [Text Editing by Command](https://mlsanity.com/paper/2010.12826/) (2020)
- [Learning to Model Editing Processes](https://mlsanity.com/paper/2205.12374/) (2022)
- [arXivEdits: Understanding the Human Revision Process in Scientific Writing](https://mlsanity.com/paper/2210.15067/) (2022)
- [A Computational Inflection for Scientific Discovery](https://mlsanity.com/paper/2205.02007/) (2022)
- [Augmenting Scientific Creativity with an Analogical Search Engine](https://mlsanity.com/paper/2205.15476/) (2022)
- [CiteSee: Augmenting Citations in Scientific Papers with Persistent and Personalized Historical Context](https://mlsanity.com/paper/2302.07302/) (2023)
- [Towards Automated Document Revision: Grammatical Error Correction, Fluency Edits, and Beyond](https://mlsanity.com/paper/2205.11484/) (2022)
- [Improving Iterative Text Revision by Learning Where to Edit from Other Revision Tasks](https://mlsanity.com/paper/2212.01350/) (2022)
- [Beyond Summarization: Designing AI Support for Real-World Expository Writing Tasks](https://mlsanity.com/paper/2304.02623/) (2023)
- [Relating Neural Text Degeneration to Exposure Bias](https://mlsanity.com/paper/2109.08705/) (2021)
- [CORWA: A Citation-Oriented Related Work Annotation Dataset](https://mlsanity.com/paper/2205.03512/) (2022)

## Built on by (most cited first, 30 of 34)

- [MARG: Multi-Agent Review Generation for Scientific Papers](https://mlsanity.com/paper/2401.04259/) (2024)
- [A Design Space for Intelligent and Interactive Writing Assistants](https://mlsanity.com/paper/2403.14117/) (2024)
- [Aligning LLM Agents by Learning Latent Preference from User Edits](https://mlsanity.com/paper/2404.15269/) (2024)
- [Large language models for automated scholarly paper review: A survey](https://mlsanity.com/paper/2501.10326/) (2025)
- [Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation](https://mlsanity.com/paper/2502.05151/) (2025)
- [What Can Natural Language Processing Do for Peer Review?](https://mlsanity.com/paper/2405.06563/) (2024)
- [Re2: A Consistency-ensured Dataset for Full-stage Peer Review and Multi-turn Rebuttal Discussions](https://mlsanity.com/paper/2505.07920/) (2025)
- [PeerQA: A Scientific Question Answering Dataset from Peer Reviews](https://mlsanity.com/paper/2502.13668/) (2025)
- [Usefulness of LLMs as an Author Checklist Assistant for Scientific Papers: NeurIPS'24 Experiment](https://mlsanity.com/paper/2411.03417/) (2024)
- [SciDQA: A Deep Reading Comprehension Dataset over Scientific Papers](https://mlsanity.com/paper/2411.05338/) (2024)
- [How Far Are AI Scientists from Changing the World?](https://mlsanity.com/paper/2507.23276/) (2025)
- [CASIMIR: A Corpus of Scientific Articles enhanced with Multiple Author-Integrated Revisions](https://mlsanity.com/paper/2403.00241/) (2024)
- [CLAIMCHECK: How Grounded are LLM Critiques of Scientific Papers?](https://mlsanity.com/paper/2503.21717/) (2025)
- [Re3: A Holistic Framework and Dataset for Modeling Collaborative Document Revision](https://mlsanity.com/paper/2406.00197/) (2024)
- [ScholaWrite: A Dataset of End-to-End Scholarly Writing Process](https://mlsanity.com/paper/2502.02904/) (2025)
- ['Quis custodiet ipsos custodes?' Who will watch the watchmen? On Detecting AI-generated peer-reviews](https://mlsanity.com/paper/2410.09770/) (2024)
- [eRevise+RF: A Writing Evaluation System for Assessing Student Essay Revisions and Providing Formative Feedback](https://mlsanity.com/paper/2501.00715/) (2025)
- [From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systems](https://mlsanity.com/paper/2503.01424/) (2025)
- [Acceleron: A Tool to Accelerate Research Ideation](https://mlsanity.com/paper/2403.04382/) (2024)
- [Large Language Models Penetration in Scholarly Writing and Peer Review](https://mlsanity.com/paper/2502.11193/) (2025)
- [Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future](https://mlsanity.com/paper/2604.27924/) (2026)
- [Author-in-the-Loop Response Generation and Evaluation: Integrating Author Expertise and Intent in Responses to Peer Review](https://mlsanity.com/paper/2602.11173/) (2026)
- [Defend: Automated Rebuttals for Peer Review with Minimal Author Guidance](https://mlsanity.com/paper/2603.27360/) (2026)
- [Efficient Layer-wise LLM Fine-tuning for Revision Intention Prediction](https://mlsanity.com/paper/2510.00268/) (2025)
- [Expos'ia: Teaching and Assessment of Academic Writing Skills for Research Project Proposals and Peer Feedback](https://mlsanity.com/paper/2601.06536/) (2026)
- [ABCD-LINK: Annotation Bootstrapping for Cross-Document Fine-Grained Links](https://mlsanity.com/paper/2509.01387/) (2025)
- [ResearchArcade: Graph Interface for Academic Tasks](https://mlsanity.com/paper/2511.22036/) (2025)
- [Principled Fine-tuning of LLMs from User-Edits: A Medley of Preference, Supervision, and Reward](https://mlsanity.com/paper/2601.19055/) (2026)
- [Intention-Adaptive LLM Fine-Tuning for Text Revision Generation](https://mlsanity.com/paper/2602.00477/) (2026)
- [Feedback-to-Rubrics: Can We Learn Expert Criteria from Inline Comments?](https://mlsanity.com/paper/2605.29857/) (2026)

## Abstract (arXiv)

We introduce the task of automatically revising scientific papers based on peer feedback and release ARIES, a dataset of review comments and their corresponding paper edits. The data is drawn from real reviewer-author interactions from computer science, and we provide labels linking each reviewer comment to the specific paper edits made by the author in response. We automatically create a high-precision silver training set, as well as an expert-labeled test set that shows high inter-annotator agreement. In experiments with 10 models covering the state of the art, we find that they struggle even to identify which edits correspond to a comment -- especially when the relationship between the edit and the comment is indirect and requires reasoning to uncover. We also extensively analyze GPT-4's ability to generate edits given a comment and the original paper. We find that it often succeeds on a superficial level, but tends to rigidly follow the wording of the feedback rather than the underlying intent, and lacks technical details compared to human-written edits.