mlsanityData to 7 Oct 2026

Zero-shot reasoning for simulating scholarly peer-review

arXiv 2510.02027 ↗First posted 2 Oct 2025cs.AI cs.ETcited by 0

Not on the map. Why?

Abstract (arXiv)

Scholarly publishing requires scalable scrutiny supported by auditable evidence. This paper presents a two-component benchmark of xPeer, the peer-review simulation engine delivered through the xPeerd.com web front-end. The operational component analyzes 352 of 500 simulation records retained under stable-task criteria across disciplines and review modes. The human-reference component releases 1,108 version-1 F1000Research manuscript records with linked human reports and applies a prespecified two-human/two-xPeer comparison. Human-review text and recommendations remained outside the generation input, and source joining occurred after xPeer outputs had been persisted. This procedure defines workflow-level review withholding; prior model exposure falls outside the recorded design. Among 802 records with exactly two human reports, 271 contained two usable xPeer reviewer fields, giving a complete-pair availability rate of 33.8%. Under deterministic extraction rules, median manuscript-level report length was 1,889 words for xPeer and 763 for humans, while median concern count was 41 and 13, respectively. xPeer reports showed higher targeting, category coverage, and executability. Human reports showed higher explicit-reasoning language, lexical manuscript attestation, taxonomy-based scientific relevance, and lower mean within-source redundancy. Cross-source lexical concern matching and recommendation agreement were low. The evidence therefore defines distinct observable review profiles and a transparent reproducibility baseline. Scientific correctness of individual concerns, autonomous editorial use, and cross-system superiority require expert adjudication and common-protocol testing. The study-level dataset and exact version-pinned reproducibility record are archived on Zenodo

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 96 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

23 of 96 references.

References · 23

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  3. 2023Large Language Modelscited by 774
  4. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  5. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  6. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  7. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 155
  8. 2022From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Modelscited by 114
  9. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 106
  10. 2016Parsing Argumentation Structures in Persuasive Essayscited by 99
  11. 2024AgentReview: Exploring Peer Review Dynamics with LLM Agentscited by 92
  12. 2023Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigationcited by 74
11 more
  1. 2025Large language models for automated scholarly paper review: A surveycited by 59
  2. 2024Automating Research Synthesis with Domain-Specific Large Language Model Fine-Tuningcited by 31
  3. 2023Large Language Models as Zero-Shot Human Models for Human-Robot Interactioncited by 30
  4. 2024SiReRAG: Indexing Similar and Related Information for Multihop Reasoningcited by 23
  5. 2023Can ChatGPT be used to generate scientific hypotheses?cited by 21
  6. 2023Improving Zero-shot Visual Question Answering via Large Language Models with Reasoning Question Promptscited by 19
  7. 2024Re3: A Holistic Framework and Dataset for Modeling Collaborative Document Revisioncited by 11
  8. 2024MALADE: Orchestration of LLM-powered Agents with Retrieval Augmented Generation for Pharmacovigilancecited by 11
  9. 2024Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document Revisionscited by 11
  10. 2024Evaluating the Predictive Capacity of ChatGPT for Academic Peer Review Outcomes Across Multiple Platformscited by 11
  11. 2025Towards an automated workflow in materials science for combining multi-modal simulative and experimental information using data mining and large language modelscited by 2

Built on by

Not cited yet.