# DISAPERE: A Dataset for Discourse Structure in Peer Review Discussions

arXiv: https://arxiv.org/abs/2110.08520 · HTML: https://mlsanity.com/paper/2110.08520/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 32 references that are in the map.
- First posted on arXiv 16 Oct 2021.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [Argumentation Mining in User-Generated Web Discourse](https://mlsanity.com/paper/1601.02403/) (2016)
- [Can We Automate Scientific Reviewing?](https://mlsanity.com/paper/2102.00176/) (2021)
- [Pretrained Language Models for Sequential Sentence Classification](https://mlsanity.com/paper/1909.04054/) (2019)
- [Argument Mining for Understanding Peer Reviews](https://mlsanity.com/paper/1903.10104/) (2019)
- [Does My Rebuttal Matter? Insights from a Major NLP Conference](https://mlsanity.com/paper/1903.11367/) (2019)
- [Fine-Grained Argument Unit Recognition and Classification](https://mlsanity.com/paper/1904.09688/) (2019)
- [Aspect-based Sentiment Analysis of Scientific Reviews](https://mlsanity.com/paper/2006.03257/) (2020)
- [Argument Mining Driven Analysis of Peer-Reviews](https://mlsanity.com/paper/2012.07743/) (2020)

## Built on by (most cited first, 30 of 36)

- [ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing](https://mlsanity.com/paper/2306.00622/) (2023)
- [LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing](https://mlsanity.com/paper/2406.16253/) (2024)
- [NLPeer: A Unified Resource for the Computational Study of Peer Review](https://mlsanity.com/paper/2211.06651/) (2022)
- [Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation](https://mlsanity.com/paper/2502.05151/) (2025)
- [What Can Natural Language Processing Do for Peer Review?](https://mlsanity.com/paper/2405.06563/) (2024)
- [Re2: A Consistency-ensured Dataset for Full-stage Peer Review and Multi-turn Rebuttal Discussions](https://mlsanity.com/paper/2505.07920/) (2025)
- [PeerQA: A Scientific Question Answering Dataset from Peer Reviews](https://mlsanity.com/paper/2502.13668/) (2025)
- [Automatic Analysis of Substantiation in Scientific Peer Reviews](https://mlsanity.com/paper/2311.11967/) (2023)
- [IRIS: Interactive Research Ideation System for Accelerating Scientific Discovery](https://mlsanity.com/paper/2504.16728/) (2025)
- [Exploring Jiu-Jitsu Argumentation for Writing Peer Review Rebuttals](https://mlsanity.com/paper/2311.03998/) (2023)
- [LazyReview A Dataset for Uncovering Lazy Thinking in NLP Peer Reviews](https://mlsanity.com/paper/2504.11042/) (2025)
- [Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedback](https://mlsanity.com/paper/2508.10795/) (2025)
- [Re3: A Holistic Framework and Dataset for Modeling Collaborative Document Revision](https://mlsanity.com/paper/2406.00197/) (2024)
- [Insights from the ICLR Peer Review and Rebuttal Process](https://mlsanity.com/paper/2511.15462/) (2025)
- [Paper2Rebuttal: A Multi-Agent Framework for Transparent Author Response Assistance](https://mlsanity.com/paper/2601.14171/) (2026)
- [Acceleron: A Tool to Accelerate Research Ideation](https://mlsanity.com/paper/2403.04382/) (2024)
- [Large Language Models Penetration in Scholarly Writing and Peer Review](https://mlsanity.com/paper/2502.11193/) (2025)
- [RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation](https://mlsanity.com/paper/2603.09723/) (2026)
- [Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future](https://mlsanity.com/paper/2604.27924/) (2026)
- [OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Models](https://mlsanity.com/paper/2505.21537/) (2025)
- [DRPG (Decompose, Retrieve, Plan, Generate): An Agentic Framework for Academic Rebuttal](https://mlsanity.com/paper/2601.18081/) (2026)
- [Identifying Aspects in Peer Reviews](https://mlsanity.com/paper/2504.06910/) (2025)
- [Are the confidence scores of reviewers consistent with the review content? Evidence from top conference proceedings in AI](https://mlsanity.com/paper/2505.15031/) (2025)
- [Author-in-the-Loop Response Generation and Evaluation: Integrating Author Expertise and Intent in Responses to Peer Review](https://mlsanity.com/paper/2602.11173/) (2026)
- [Defend: Automated Rebuttals for Peer Review with Minimal Author Guidance](https://mlsanity.com/paper/2603.27360/) (2026)
- [Decision-Making with Deliberation: Meta-reviewing as a Document-grounded Dialogue](https://mlsanity.com/paper/2508.05283/) (2025)
- [Beyond Rating: A Comprehensive Evaluation and Benchmark for AI Reviews](https://mlsanity.com/paper/2604.19502/) (2026)
- [What Makes a Good AI Review? Concern-Level Diagnostics for AI Peer Review](https://mlsanity.com/paper/2604.19998/) (2026)
- [Towards Comprehensive Argument Analysis in Education: Dataset, Tasks, and Method](https://mlsanity.com/paper/2505.12028/) (2025)
- [What happens when reviewers receive AI feedback in their reviews?](https://mlsanity.com/paper/2602.13817/) (2026)

## Abstract (arXiv)

At the foundation of scientific evaluation is the labor-intensive process of peer review. This critical task requires participants to consume vast amounts of highly technical text. Prior work has annotated different aspects of review argumentation, but discourse relations between reviews and rebuttals have yet to be examined. We present DISAPERE, a labeled dataset of 20k sentences contained in 506 review-rebuttal pairs in English, annotated by experts. DISAPERE synthesizes label sets from prior work and extends them to include fine-grained annotation of the rebuttal sentences, characterizing their context in the review and the authors' stance towards review arguments. Further, we annotate every review and rebuttal sentence. We show that discourse cues from rebuttals can shed light on the quality and interpretation of reviews. Further, an understanding of the argumentative strategies employed by the reviewers and authors provides useful signal for area chairs and other decision makers.