# AI scientists produce results without reasoning scientifically

arXiv: https://arxiv.org/abs/2604.18805 · HTML: https://mlsanity.com/paper/2604.18805/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- First posted on arXiv 20 Apr 2026.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Built on by (most cited first, 17 of 17)

- [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026)
- [DiscoverPhysics: Benchmarking LLMs for Out-of-the-Box Scientific Thinking](https://mlsanity.com/paper/2605.26087/) (2026)
- [Agentic AI Scientists Are Not Built For Autonomous Scientific Discovery](https://mlsanity.com/paper/2605.08956/) (2026)
- [MDGYM: Benchmarking AI Agents on Molecular Simulations](https://mlsanity.com/paper/2605.08941/) (2026)
- [Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents](https://mlsanity.com/paper/2607.09195/) (2026)
- [Discovery Foundation Models: Toward Open-Ended Discovery Intelligence](https://mlsanity.com/paper/2609.15973/) (2026)
- [Distributed Denial of Science: How Indirect Data Poisoning of AI Systems Can Industrialize Scientific Fraud](https://mlsanity.com/paper/2607.10712/) (2026)
- [EvoSCM: Scientific Belief Revision Through Causal Model Evolution and Experimentation](https://mlsanity.com/paper/2609.01526/) (2026)
- [Heterogeneous Scientific Foundation Model Collaboration](https://mlsanity.com/paper/2604.27351/) (2026)
- [Workflow Closure Is Not Scientific Closure in Auto-Research Systems](https://mlsanity.com/paper/2605.26200/) (2026)
- [A Pathway to General-Purpose Scientific AI: Multimodal Comprehension of Scientific Images](https://mlsanity.com/paper/2608.14075/) (2026)
- [MathAdv: What Theorem Provers Know, Reason, Formalize, and Generalize](https://mlsanity.com/paper/2608.25449/) (2026)
- [Metacognitive Steering: Learning the Structure of Scientific Judgment](https://mlsanity.com/paper/2609.16245/) (2026)
- [Research-Native by Construction: Minimal Nodes, Re-verifiable Workflows, and Compounding Memory for Long-Horizon Scientific Agents](https://mlsanity.com/paper/2609.35182/) (2026)
- [Predictive Credit: Measuring What Scientific Explanations Add to Experimental Forecasts](https://mlsanity.com/paper/2610.00314/) (2026)
- [Triage-PiEvo: Evidence Admission for Principle-Evolving Scientific Agents](https://mlsanity.com/paper/2610.14683/) (2026)
- [Big-Big-SycoBench: A Benchmark to Detect Sycophancy Through Model Responses to Valid vs. Invalid Arguments](https://mlsanity.com/paper/2610.15634/) (2026)

## Abstract (arXiv)

Large language model (LLM)-based systems are increasingly deployed to conduct scientific research autonomously, yet whether their reasoning adheres to the epistemic norms that make scientific inquiry self-correcting is poorly understood. Here, we evaluate LLM-based scientific agents across eight domains, spanning workflow execution to hypothesis-driven inquiry, through more than 25,000 agent runs and two complementary lenses: (i) a systematic performance analysis that decomposes the contributions of the base model and the agent scaffold, and (ii) a behavioral analysis of the epistemological structure of agent reasoning. We observe that the base model is the primary determinant of both performance and behavior, accounting for 41.4% of explained variance versus 1.5% for the scaffold. Across all configurations, evidence is ignored in 68% of traces, refutation-driven belief revision occurs in 26%, and convergent multi-test evidence is rare. The same reasoning pattern appears whether the agent executes a computational workflow or conducts hypothesis-driven inquiry. They persist even when agents receive near-complete successful reasoning trajectories as context, and the resulting unreliability compounds across repeated trials in epistemically demanding domains. Thus, current LLM-based agents execute scientific workflows but do not exhibit the epistemic patterns that characterize scientific reasoning. Outcome-based evaluation cannot detect these failures, and scaffold engineering alone cannot repair them. Until reasoning itself becomes a training target, the scientific knowledge produced by such agents cannot be justified by the process that generated it.