Agentic AI Scientists Are Not Built For Autonomous Scientific Discovery
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
A growing body of work pursues AI scientists capable of end-to-end autonomous scientific discovery. This position paper argues that although they already function as co-scientists, agentic AI scientists are not built for autonomous scientific discovery. We identify the following challenges in building and deploying autonomous AI scientists: (1) Problem selection is influenced by the McNamara fallacy; (2) Agents are built on large language models (LLMs) whose training corpora omit tacit procedural and failure knowledge of laboratory practice; (3) Preference optimisation during post-training compresses output diversity toward consensus; and (4) Most scientific benchmarks measure single-turn prediction accuracy and lack feedback from physical experiments back to the computational model. These challenges are not just questions of scale and scaffolding; they require revisiting fundamental design choices. To build truly autonomous AI scientists, we recommend scientific simulations as verifiers for training, a persistent, mutable epistemic state that carries beliefs and shifting objectives across an investigation, the establishment of a centralized preregistration repository for all AI-generated hypotheses, and application driven by scientific need rather than tool affordance.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 39 of its 188 references that are in the map. First posted on arXiv 9 May 2026.
Builds on
39 of 188 references.
Closest ancestors · 1
Same direction, newest first
- 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search10 Apr
Other references · 38
- 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
- 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 701
- 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 467
- 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
- 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 342
- 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 299
- 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 286
- 2026Towards End-to-End Automation of AI Researchcited by 150
- 2025Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)cited by 137
- 2024MaxMin-RLHF: Alignment with Diverse Human Preferencescited by 121
- 2025The Art of Scaling Reinforcement Learning Compute for LLMscited by 91
26 more
- 2025Kosmos: An AI Scientist for Autonomous Discoverycited by 78
- 2024ORGANA: A Robotic Assistant for Automated Chemistry Experimentation and Characterizationcited by 60
- 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 57
- 2025Robin: A multi-agent system for automating scientific discoverycited by 57
- 2026MIRAGE: The Illusion of Visual Understandingcited by 57
- 2023OpenMM 8: Molecular Dynamics Simulation with Machine Learning Potentialscited by 44
- 2024Divergent Creativity in Humans and Large Language Modelscited by 29
- 2013Tradition and Innovation in Scientists' Research Strategiescited by 26
- 2020Autonomous discovery in the chemical sciences part II: Outlookcited by 26
- 2020Autonomous discovery in the chemical sciences part I: Progresscited by 25
- 2024No Preference Left Behind: Group Distributional Preference Optimizationcited by 21
- 2025Exploring the use of AI authors and reviewers at Agents4Sciencecited by 13
- 2026AI scientists produce results without reasoning scientificallycited by 13
- 2025Universally Converging Representations of Matter Across Scientific Foundation Modelscited by 12
- 2025GEOM-Drugs Revisited: Toward More Chemically Accurate Benchmarks for 3D Molecule Generationcited by 8
- 2019Quantifying dynamics of failure across science, startups, and securitycited by 4
- 2025MatPROV: A Provenance Graph Dataset of Material Synthesis Extracted from Scientific Literaturecited by 3
- 2025Towards Fully Automated Molecular Simulations: Multi-Agent Framework for Simulation Setup and Force Field Extractioncited by 3
- 2026Solving Physics Olympiad via Reinforcement Learning on Physics Simulatorscited by 3
- 2025Training speedups via batching for geometric learning: an analysis of static and dynamic algorithmscited by 2
- 2025Surrogate Modeling for the Design of Optimal Lattice Structures using Tensor Completioncited by 2
- 2025LLM Agents for Knowledge Discovery in Atomic Layer Processingcited by 1
- 2025Benchmarking Agentic Systems in Automated Scientific Information Extraction with ChemXcited by 1
- 2025Pharmacophore-Guided Generative Design of Novel Drug-Like Moleculescited by 1
- 2025Preference Learning from Physics-Based Feedback: Tuning Language Models to Design BCC/B2 Superalloyscited by 1
- 2025Efficient Nudged Elastic Band Method using Neural Network Bayesian Algorithm Executioncited by 1
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loopscited by 25
- 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 6
- 2026Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world modelscited by 6
- 2026How Do Agents Fail on AutoResearch: End-to-End Diagnostic Evaluation on 100 Real-World Frontier Research Taskscited by 4
- 2026Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agentscited by 3
- 2026Socratic agents for autonomous scientific discovery in high-dimensional physical systemscited by 2
- 2026Evolutionary Safety of Recursive Self-Improving AI: Taxonomy, Risk Discovery, and Evaluationcited by 0
- 2026Evolutionary Safety of Recursive Self-Improving AI: Taxonomy, Risk Discovery, and Evaluation25 Sep 2026
- 2026How Do Agents Fail on AutoResearch: End-to-End Diagnostic Evaluation on 100 Real-World Frontier Research Tasks14 Aug 2026
- 2026Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models10 Aug 2026
- 2026Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents10 Jul 2026
- 2026Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops8 Jul 2026
- 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap29 Jun 2026
- 2026Socratic agents for autonomous scientific discovery in high-dimensional physical systems25 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery
- 2025AI-Researcher: Autonomous Scientific Innovation
- 2026AI Scientists as Engines of Discovery: A Case for Development within Reformed Institutions
- 2026Why LLMs Aren't Scientists Yet: Lessons from Four Autonomous Research Attempts
- 2026Position: AI Agents in Scientific Teams Should Be Studied as Human-Agent Systems
- 2025Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physics
- 2025Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions
- 2025Virtuous Machines: Towards Artificial General Science