# PeeriScope: A Multi-Faceted Framework for Evaluating Peer Review Quality

arXiv: https://arxiv.org/abs/2604.24071 · HTML: https://mlsanity.com/paper/2604.24071/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 27 Apr 2026.

## Builds on (most cited first, 9 of 9)

- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [SPECTER: Document-level Representation Learning using Citation-informed Transformers](https://mlsanity.com/paper/2004.07180/) (2020)
- [LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing](https://mlsanity.com/paper/2406.16253/) (2024)
- [Summarizing Multiple Documents with Conversational Structure for Meta-Review Generation](https://mlsanity.com/paper/2305.01498/) (2023)
- [Evaluating and Enhancing Large Language Models for Novelty Assessment in Scholarly Publications](https://mlsanity.com/paper/2409.16605/) (2024)
- [Breaking the Reviewer: Assessing the Vulnerability of Large Language Models in Automated Peer Review Under Textual Adversarial Attacks](https://mlsanity.com/paper/2506.11113/) (2025)
- [LazyReview A Dataset for Uncovering Lazy Thinking in NLP Peer Reviews](https://mlsanity.com/paper/2504.11042/) (2025)
- [exHarmony: Authorship and Citations for Benchmarking the Reviewer Assignment Problem](https://mlsanity.com/paper/2502.07683/) (2025)
- [Disparities in Peer Review Tone and the Role of Reviewer Anonymity](https://mlsanity.com/paper/2507.14741/) (2025)

## Built on by (most cited first, 2 of 2)

- [Judging a Review by its Cover: A Reliability Analysis of LLM-based Peer Review Evaluation Metrics](https://mlsanity.com/paper/2609.23264/) (2026)
- [Peerify: Benchmarking Peer-Review Claim Verification](https://mlsanity.com/paper/2609.25046/) (2026)

## Abstract (arXiv)

The increasing scale and variability of peer review in scholarly venues has created an urgent need for systematic, interpretable, and extensible tools to assess review quality. We present PeeriScope, a modular platform that integrates structured features, rubric-guided large language model assessments, and supervised prediction to evaluate peer review quality along multiple dimensions. Designed for openness and integration, PeeriScope provides both a public interface and a documented API, supporting practical deployment and research extensibility. The demonstration illustrates its use for reviewer self-assessment, editorial triage, and large-scale auditing, and it enables the continued development of quality evaluation methods within scientific peer review. PeeriScope is available both as a live demo at https://app.reviewer.ly/app/peeriscope and via API services at https://github.com/Reviewerly-Inc/Peeriscope.