# When AI Co-Scientists Fail: SPOT-a Benchmark for Automated Verification of Scientific Research

arXiv: https://arxiv.org/abs/2505.11855 · HTML: https://mlsanity.com/paper/2505.11855/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 74 references that are in the map.
- First posted on arXiv 17 May 2025.

## Builds on (most cited first, 30 of 50)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [ChemCrow: Augmenting large-language models with chemistry tools](https://mlsanity.com/paper/2304.05376/) (2023)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [Accelerating scientific discovery with Co-Scientist](https://mlsanity.com/paper/2502.18864/) (2025)
- [Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers](https://mlsanity.com/paper/2409.04109/) (2024)
- [PaperBench: Evaluating AI's Ability to Replicate AI Research](https://mlsanity.com/paper/2504.01848/) (2025)
- [SPoC: Search-based Pseudocode to Code](https://mlsanity.com/paper/1906.04908/) (2019)
- [CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark](https://mlsanity.com/paper/2409.11363/) (2024)
- [Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries](https://mlsanity.com/paper/2409.12640/) (2024)
- [MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures](https://mlsanity.com/paper/2406.06565/) (2024)
- [Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad](https://mlsanity.com/paper/2503.21934/) (2025)
- [Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning](https://mlsanity.com/paper/2504.17192/) (2025)
- [PaSa: An LLM Agent for Comprehensive Academic Paper Search](https://mlsanity.com/paper/2501.10120/) (2025)
- [Scaling Scaling Laws with Board Games](https://mlsanity.com/paper/2104.03113/) (2021)

## Built on by (most cited first, 26 of 26)

- [ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents](https://mlsanity.com/paper/2511.07685/) (2025)
- [The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems](https://mlsanity.com/paper/2509.08713/) (2025)
- [FLAWS: A Benchmark for Error Identification and Localization in Scientific Papers](https://mlsanity.com/paper/2511.21843/) (2025)
- [Reviewing Scientific Papers for Critical Problems With Reasoning LLMs: Baseline Approaches and Automatic Evaluation](https://mlsanity.com/paper/2505.23824/) (2025)
- [DEER: A Benchmark for Evaluating Deep Research Agents on Expert Report Generation](https://mlsanity.com/paper/2512.17776/) (2025)
- [Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Framework](https://mlsanity.com/paper/2508.21422/) (2025)
- [A Survey of AI Scientists](https://mlsanity.com/paper/2510.23045/) (2025)
- [SciCoQA: Quality Assurance for Scientific Paper--Code Alignment](https://mlsanity.com/paper/2601.12910/) (2026)
- [AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discovery](https://mlsanity.com/paper/2605.23204/) (2026)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Agon: An Autonomous Large-Scale Omnidisciplinary Research System Built on Prompt Economy](https://mlsanity.com/paper/2606.24177/) (2026)
- [PaperAudit-Bench: Benchmarking Error Detection in Research Papers for Critical Automated Peer Review](https://mlsanity.com/paper/2601.19916/) (2026)
- [Grounded autonomous scrutiny at scale: emergent critique from reproduction of published computational physics papers](https://mlsanity.com/paper/2604.12198/) (2026)
- [ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues](https://mlsanity.com/paper/2606.18237/) (2026)
- [Towards Automating Scientific Review with Google's Paper Assistant Tool](https://mlsanity.com/paper/2606.28277/) (2026)
- [Judging What We Cannot Solve: A Consequence-Based Approach for Oracle-Free Evaluation of Research-Level Math](https://mlsanity.com/paper/2602.06291/) (2026)
- [The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing](https://mlsanity.com/paper/2609.07713/) (2026)
- [On Finding Inconsistencies in Documents](https://mlsanity.com/paper/2512.18601/) (2025)
- [Toward an Engineering of Science: Rebalancing Generation and Verification in the Age of AI](https://mlsanity.com/paper/2605.10425/) (2026)
- [Automatic Ordinary Differential Equations Discovery For Biological Systems Using Large Language Model Powered Agentic System](https://mlsanity.com/paper/2607.13608/) (2026)
- [Personalized Auto-Research: Towards a True AI Co-Scientist](https://mlsanity.com/paper/2608.14881/) (2026)
- [When Auditors Fabricate: Batch-Size Degradation and Confident Hallucination in LLM Detection of Planted Document Contamination](https://mlsanity.com/paper/2609.09696/) (2026)
- [When Scientific Contradictions Are Lost in Translation](https://mlsanity.com/paper/2609.38621/) (2026)
- [Beyond Imitation: A Framework and Benchmark for LLM-Assisted Peer Review](https://mlsanity.com/paper/2610.11087/) (2026)
- [How Is Automated Research Evaluated? A Survey of Benchmarks and Evaluation Practices](https://mlsanity.com/paper/2610.11877/) (2026)
- [One-Sided Verification Under Limited Expert Review](https://mlsanity.com/paper/2610.18089/) (2026)

## Abstract (arXiv)

Recent advances in large language models (LLMs) have fueled the vision of automated scientific discovery, often called AI Co-Scientists. To date, prior work casts these systems as generative co-authors responsible for crafting hypotheses, synthesizing code, or drafting manuscripts. In this work, we explore a complementary application: using LLMs as verifiers to automate the academic verification of scientific manuscripts. To that end, we introduce SPOT, a dataset of 83 published papers paired with 91 errors significant enough to prompt errata or retraction, cross-validated with actual authors and human annotators. Evaluating state-of-the-art LLMs on SPOT, we find that none surpasses 21.1% recall or 6.1% precision (o3 achieves the best scores, with all others near zero). Furthermore, confidence estimates are uniformly low, and across eight independent runs, models rarely rediscover the same errors, undermining their reliability. Finally, qualitative analysis with domain experts reveals that even the strongest models make mistakes resembling student-level misconceptions derived from misunderstandings. These findings highlight the substantial gap between current LLM capabilities and the requirements for dependable AI-assisted academic verification.