mlsanityData to 7 Oct 2026

FactReview: Evidence-Grounded Peer Review with Execution-Based Claim Verification

arXiv 2604.04074 ↗First posted 5 Apr 2026cs.AI cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Large language model (LLM)-based reviewing systems typically assess manuscripts in isolation, leaving literature- and code-dependent claims difficult to verify. We present FactReview, an audit pipeline that extracts review-relevant claims, grounds them in related work and reference checks, and, when code is available, executes released artifacts under a fixed repair budget. On 26 paper-disjoint test papers with 354 human-verified claims, FactReview achieves 84.3% F1 for claim recovery. In a same-backend, evidence-matched comparison, FactReview scores 4.72/5 overall, outperforming a direct LLM reviewer by 0.74 points. Removing execution evidence changes 17.0% of claim statuses, more than removing any other single evidence source. In a reviewer-assistance study, FactReview reduces mean review time by 58% while increasing benchmark-claim coverage from 87% to 99%. FactReview supports evidence-based claim auditing, with acceptance decisions reserved for human reviewers. The code is public at https://github.com/DEFENSE-SEU/FactReview.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 5 Apr 2026.

Builds on

32 of 50 references.

References · 32

  1. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  2. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  3. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  4. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  5. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
  6. 2019Composition-based Multi-Relational Graph Convolutional Networkscited by 395
  7. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 339
  8. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 318
  9. 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 286
  10. 2023Can large language models provide useful feedback on research papers? A large-scale empirical analysiscited by 195
  11. 2022OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and conceptscited by 166
  12. 2023PaperQA: Retrieval-Augmented Generative Agent for Scientific Researchcited by 143
20 more
  1. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  2. 2018End-to-end Structure-Aware Convolutional Networks for Knowledge Base Completioncited by 136
  3. 2024CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmarkcited by 115
  4. 2024MARG: Multi-Agent Review Generation for Scientific Paperscited by 98
  5. 2025DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Processcited by 97
  6. 2024AgentReview: Exploring Peer Review Dynamics with LLM Agentscited by 92
  7. 2019A Step Toward Quantifying Independently Reproducible Machine Learning Researchcited by 77
  8. 2024OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMscited by 64
  9. 2025Large language models for automated scholarly paper review: A surveycited by 59
  10. 2019BERT for Evidence Retrieval and Claim Verificationcited by 57
  11. 2025ResearchCodeBench: Benchmarking LLMs on Implementing Novel Machine Learning Research Codecited by 34
  12. 2017Design and Analysis of the NIPS 2016 Review Processcited by 33
  13. 2025AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineagecited by 29
  14. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoningcited by 25
  15. 2020A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferencescited by 16
  16. 2025When Your Reviewer is an LLM: Biases, Divergence, and Prompt Injection Risks in Peer Reviewcited by 16
  17. 2025ReviewerToo: Should AI Join The Program Committee? A Look At The Future of Peer Reviewcited by 12
  18. 2025CLAIMCHECK: How Grounded are LLM Critiques of Scientific Papers?cited by 11
  19. 2025LLM-based Corroborating and Refuting Evidence Retrieval for Scientific Claim Verificationcited by 5
  20. 2024MLXP: A Framework for Conducting Replicable Experiments in Pythoncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026SCIRIGOR:Evaluating Open-Ended Scientific Analysis Beyond Final Scorescited by 0
  2. 2026When Evidence Changes the Subject: Subject-Typed Claim Licensing for Learned Routingcited by 0