# A Missing Piece for Trustworthy AI Reviewers: From Benchmarking Rhetorical Robustness to SciCore Review

arXiv: https://arxiv.org/abs/2609.39027 · HTML: https://mlsanity.com/paper/2609.39027/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 46 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Builds on (most cited first, 30 of 33)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators](https://mlsanity.com/paper/2404.04475/) (2024)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks](https://mlsanity.com/paper/2406.18403/) (2024)
- [Large Language Models are Inconsistent and Biased Evaluators](https://mlsanity.com/paper/2405.01724/) (2024)
- [A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications](https://mlsanity.com/paper/1804.09635/) (2018)
- [DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Process](https://mlsanity.com/paper/2503.08569/) (2025)
- [OpenReviewer: A Specialized Large Language Model for Generating Critical Scientific Paper Reviews](https://mlsanity.com/paper/2412.11948/) (2024)
- [Can We Automate Scientific Reviewing?](https://mlsanity.com/paper/2102.00176/) (2021)
- [ReviewRobot: Explainable Paper Review Generation based on Knowledge Synthesis](https://mlsanity.com/paper/2010.06119/) (2020)
- [Lossy Compression for Lossless Prediction](https://mlsanity.com/paper/2106.10800/) (2021)
- [Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the effect of Epistemic Markers on LLM-based Evaluation](https://mlsanity.com/paper/2410.20774/) (2024)
- [When Your Reviewer is an LLM: Biases, Divergence, and Prompt Injection Risks in Peer Review](https://mlsanity.com/paper/2509.09912/) (2025)
- [Breaking the Reviewer: Assessing the Vulnerability of Large Language Models in Automated Peer Review Under Textual Adversarial Attacks](https://mlsanity.com/paper/2506.11113/) (2025)
- [Stop Automating Peer Review Without Rigorous Evaluation](https://mlsanity.com/paper/2605.03202/) (2026)
- [Unveiling the Merits and Defects of LLMs in Automatic Review Generation for Scientific Papers](https://mlsanity.com/paper/2509.19326/) (2025)
- [Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Framework](https://mlsanity.com/paper/2508.21422/) (2025)
- [Justice in Judgment: Unveiling (Hidden) Bias in LLM-assisted Peer Reviews](https://mlsanity.com/paper/2509.13400/) (2025)
- [Paraphrasing Adversarial Attack on LLM-as-a-Reviewer](https://mlsanity.com/paper/2601.06884/) (2026)
- [Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry](https://mlsanity.com/paper/2601.22588/) (2026)
- [Misleading Large Language Models used (or misused) in Scientific Peer-Reviewing via Hidden Prompt-Injection Attacks](https://mlsanity.com/paper/2508.20863/) (2025)
- [On the Rigour of Scientific Writing: Criteria, Analysis, and Insights](https://mlsanity.com/paper/2410.04981/) (2024)
- [What Makes a Scientific Paper be Accepted for Publication?](https://mlsanity.com/paper/2104.07112/) (2021)
- [Turning Bias into Bugs: Bandit-Guided Style Manipulation Attacks on LLM Judges](https://mlsanity.com/paper/2605.26156/) (2026)
- [From Passive Generation to Investigation: A Proactive Scientific Peer Review Agent](https://mlsanity.com/paper/2606.13349/) (2026)
- [Gaming AI-Assisted Peer Reviews Poses New Risks to the Scientific Community](https://mlsanity.com/paper/2606.10159/) (2026)
- [PeerCheck: Enhancing LLM-Generated Academic Reviews Towards Human-Level Quality](https://mlsanity.com/paper/2606.20897/) (2026)

## Abstract (arXiv)

AI reviewers can assign different judgments to manuscripts that report the same science in different wording, potentially rewarding rhetorical optimization over scientific improvement. We formulate Rhetorical Robustness as the joint requirement of stability across content-preserving rewrites and discrimination across papers. We introduce RobustReview, a controlled full-manuscript benchmark with 1,260 manuscript versions, and evaluate 30 reviewer configurations. The benchmark reveals false robustness, where low rewrite sensitivity coincides with score collapse across papers, and shows that human alignment and rhetorical robustness rank reviewers differently. Moreover, the evaluated content-focused prompting protocol does not consistently improve robustness across backbones. Motivated by these findings, we introduce SciCore, a dual-branch reviewer that averages a full-manuscript judgment with a judgment based on an extracted, structured science core. This design combines manuscript-level assessment with a content-normalized view intended to reduce rhetorical sensitivity. In our primary GPT-5.5 comparison, SciCore achieves a leading joint stability-discrimination profile among the benchmarked reviewers while maintaining competitive human alignment. These results identify rhetorical robustness as a distinct evaluation target and demonstrate the potential of science-core review to improve it.