mlsanityData to 7 Oct 2026

DISAPERE: A Dataset for Discourse Structure in Peer Review Discussions

arXiv 2110.08520 ↗First posted 16 Oct 2021cs.CLcited by 35

Not on the map. Why?

Abstract (arXiv)

At the foundation of scientific evaluation is the labor-intensive process of peer review. This critical task requires participants to consume vast amounts of highly technical text. Prior work has annotated different aspects of review argumentation, but discourse relations between reviews and rebuttals have yet to be examined. We present DISAPERE, a labeled dataset of 20k sentences contained in 506 review-rebuttal pairs in English, annotated by experts. DISAPERE synthesizes label sets from prior work and extends them to include fine-grained annotation of the rebuttal sentences, characterizing their context in the review and the authors' stance towards review arguments. Further, we annotate every review and rebuttal sentence. We show that discourse cues from rebuttals can shed light on the quality and interpretation of reviews. Further, an understanding of the argumentative strategies employed by the reviewers and authors provides useful signal for area chairs and other decision makers.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 13 of its 32 references that are in the map. First posted on arXiv 16 Oct 2021.

Builds on

13 of 32 references.

References · 13

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  4. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  5. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  6. 2016Argumentation Mining in User-Generated Web Discoursecited by 69
  7. 2021Can We Automate Scientific Reviewing?cited by 63
  8. 2019Pretrained Language Models for Sequential Sentence Classificationcited by 53
  9. 2019Argument Mining for Understanding Peer Reviewscited by 41
  10. 2019Does My Rebuttal Matter? Insights from a Major NLP Conferencecited by 31
  11. 2019Fine-Grained Argument Unit Recognition and Classificationcited by 26
  12. 2020Aspect-based Sentiment Analysis of Scientific Reviewscited by 24
1 more
  1. 2020Argument Mining Driven Analysis of Peer-Reviewscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 4; 2025: 13; 2026: 13.

35 papers.

Sort:

By area:

  1. 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 118
  2. 2024LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewingcited by 62
  3. 2022NLPeer: A Unified Resource for the Computational Study of Peer Reviewcited by 60
  4. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 41
  5. 2024What Can Natural Language Processing Do for Peer Review?cited by 24
  6. 2025PeerQA: A Scientific Question Answering Dataset from Peer Reviewscited by 21
  7. 2023Automatic Analysis of Substantiation in Scientific Peer Reviewscited by 19
  8. 2025Re2: A Consistency-ensured Dataset for Full-stage Peer Review and Multi-turn Rebuttal Discussionscited by 19
  9. 2023Exploring Jiu-Jitsu Argumentation for Writing Peer Review Rebuttalscited by 18
  10. 2025IRIS: Interactive Research Ideation System for Accelerating Scientific Discoverycited by 16
  11. 2025LazyReview A Dataset for Uncovering Lazy Thinking in NLP Peer Reviewscited by 13
  12. 2025Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedbackcited by 12
23 more
  1. 2024Re3: A Holistic Framework and Dataset for Modeling Collaborative Document Revisioncited by 11
  2. 2025Insights from the ICLR Peer Review and Rebuttal Processcited by 8
  3. 2026Paper2Rebuttal: A Multi-Agent Framework for Transparent Author Response Assistancecited by 8
  4. 2024Acceleron: A Tool to Accelerate Research Ideationcited by 6
  5. 2025Large Language Models Penetration in Scholarly Writing and Peer Reviewcited by 6
  6. 2026Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Futurecited by 6
  7. 2025OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Modelscited by 5
  8. 2026RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generationcited by 5
  9. 2025Identifying Aspects in Peer Reviewscited by 4
  10. 2025Are the confidence scores of reviewers consistent with the review content? Evidence from top conference proceedings in AIcited by 4
  11. 2026DRPG (Decompose, Retrieve, Plan, Generate): An Agentic Framework for Academic Rebuttalcited by 4
  12. 2025Decision-Making with Deliberation: Meta-reviewing as a Document-grounded Dialoguecited by 3
  13. 2026Author-in-the-Loop Response Generation and Evaluation: Integrating Author Expertise and Intent in Responses to Peer Reviewcited by 3
  14. 2026Defend: Automated Rebuttals for Peer Review with Minimal Author Guidancecited by 3
  15. 2026Beyond Rating: A Comprehensive Evaluation and Benchmark for AI Reviewscited by 3
  16. 2026What Makes a Good AI Review? Concern-Level Diagnostics for AI Peer Reviewcited by 2
  17. 2025Towards Comprehensive Argument Analysis in Education: Dataset, Tasks, and Methodcited by 1
  18. 2026What happens when reviewers receive AI feedback in their reviews?cited by 1
  19. 2026AutoSupervision: Closing the Feedback Loop in Scientific Workflows with Grounded Revision Verificationcited by 1
  20. 2022An Inclusive Notion of Textcited by 0
  21. 2026Reviewing the Reviewer: LLM-Assisted Reviewer Feedback Generation for Guideline Compliancecited by 0
  22. 2026Rebuttals Move Peer-Review Scores, but Initial-Review Structure Bounds the Movementcited by 0
  23. 2026ReGround: Grounding Reviewer Comments in Multimodal Evidencecited by 0