# Agentic AI Scientists Are Not Built For Autonomous Scientific Discovery arXiv: https://arxiv.org/abs/2605.08956 ยท HTML: https://mlsanity.com/paper/2605.08956/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. - Builds on 39 of its 188 references that are in the map. - First posted on arXiv 9 May 2026. ## Placed in - LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/) ## Builds on (most cited first, 30 of 39) - [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025) - [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018) - [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021) - [MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields](https://mlsanity.com/paper/2206.07697/) (2022) - [The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search](https://mlsanity.com/paper/2504.08066/) (2025) - [Understanding the Effects of RLHF on LLM Generalisation and Diversity](https://mlsanity.com/paper/2310.06452/) (2023) - [Noise or Signal: The Role of Image Backgrounds in Object Recognition](https://mlsanity.com/paper/2006.09994/) (2020) - [PaperBench: Evaluating AI's Ability to Replicate AI Research](https://mlsanity.com/paper/2504.01848/) (2025) - [The Vendi Score: A Diversity Evaluation Metric for Machine Learning](https://mlsanity.com/paper/2210.02410/) (2022) - [Towards End-to-End Automation of AI Research](https://mlsanity.com/paper/2606.15497/) (2026) - [Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)](https://mlsanity.com/paper/2510.22954/) (2025) - [MaxMin-RLHF: Alignment with Diverse Human Preferences](https://mlsanity.com/paper/2402.08925/) (2024) - [The Art of Scaling Reinforcement Learning Compute for LLMs](https://mlsanity.com/paper/2510.13786/) (2025) - [Kosmos: An AI Scientist for Autonomous Discovery](https://mlsanity.com/paper/2511.02824/) (2025) - [MIRAGE: The Illusion of Visual Understanding](https://mlsanity.com/paper/2603.21687/) (2026) - [Robin: A multi-agent system for automating scientific discovery](https://mlsanity.com/paper/2505.13400/) (2025) - [ORGANA: A Robotic Assistant for Automated Chemistry Experimentation and Characterization](https://mlsanity.com/paper/2401.06949/) (2024) - [Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluation](https://mlsanity.com/paper/2502.06559/) (2025) - [OpenMM 8: Molecular Dynamics Simulation with Machine Learning Potentials](https://mlsanity.com/paper/2310.03121/) (2023) - [Divergent Creativity in Humans and Large Language Models](https://mlsanity.com/paper/2405.13012/) (2024) - [Tradition and Innovation in Scientists' Research Strategies](https://mlsanity.com/paper/1302.6906/) (2013) - [Autonomous discovery in the chemical sciences part II: Outlook](https://mlsanity.com/paper/2003.13755/) (2020) - [Autonomous discovery in the chemical sciences part I: Progress](https://mlsanity.com/paper/2003.13754/) (2020) - [No Preference Left Behind: Group Distributional Preference Optimization](https://mlsanity.com/paper/2412.20299/) (2024) - [AI scientists produce results without reasoning scientifically](https://mlsanity.com/paper/2604.18805/) (2026) - [Universally Converging Representations of Matter Across Scientific Foundation Models](https://mlsanity.com/paper/2512.03750/) (2025) - [Exploring the use of AI authors and reviewers at Agents4Science](https://mlsanity.com/paper/2511.15534/) (2025) - [GEOM-Drugs Revisited: Toward More Chemically Accurate Benchmarks for 3D Molecule Generation](https://mlsanity.com/paper/2505.00169/) (2025) - [Quantifying dynamics of failure across science, startups, and security](https://mlsanity.com/paper/1903.07562/) (2019) - [Solving Physics Olympiad via Reinforcement Learning on Physics Simulators](https://mlsanity.com/paper/2604.11805/) (2026) ## Built on by (most cited first, 8 of 8) - [Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops](https://mlsanity.com/paper/2607.07663/) (2026) - [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026) - [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026) - [How Do Agents Fail on AutoResearch: End-to-End Diagnostic Evaluation on 100 Real-World Frontier Research Tasks](https://mlsanity.com/paper/2608.14905/) (2026) - [Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models](https://mlsanity.com/paper/2608.09696/) (2026) - [Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents](https://mlsanity.com/paper/2607.09195/) (2026) - [Socratic agents for autonomous scientific discovery in high-dimensional physical systems](https://mlsanity.com/paper/2606.26722/) (2026) - [Evolutionary Safety of Recursive Self-Improving AI: Taxonomy, Risk Discovery, and Evaluation](https://mlsanity.com/paper/2609.31186/) (2026) ## Abstract (arXiv) A growing body of work pursues AI scientists capable of end-to-end autonomous scientific discovery. This position paper argues that although they already function as co-scientists, agentic AI scientists are not built for autonomous scientific discovery. We identify the following challenges in building and deploying autonomous AI scientists: (1) Problem selection is influenced by the McNamara fallacy; (2) Agents are built on large language models (LLMs) whose training corpora omit tacit procedural and failure knowledge of laboratory practice; (3) Preference optimisation during post-training compresses output diversity toward consensus; and (4) Most scientific benchmarks measure single-turn prediction accuracy and lack feedback from physical experiments back to the computational model. These challenges are not just questions of scale and scaffolding; they require revisiting fundamental design choices. To build truly autonomous AI scientists, we recommend scientific simulations as verifiers for training, a persistent, mutable epistemic state that carries beliefs and shifting objectives across an investigation, the establishment of a centralized preregistration repository for all AI-generated hypotheses, and application driven by scientific need rather than tool affordance.