Workflow Closure Is Not Scientific Closure in Auto-Research Systems
Not on the map. Why?
Abstract (arXiv)
This paper argues that workflow closure is not scientific closure in auto-research systems. Current systems can increasingly complete research-like loops internally, moving from idea generation to experiment execution, writing, and self-evaluation. That achievement is real, but it does not by itself give the resulting outputs scientific standing. We argue that trustworthy auto-research should not aim for autonomous self-sufficiency, but should aim for autonomous execution under non-autonomous epistemic control. Based on a survey of more than 100 recent papers and repositories in this rapidly emerging area, together with a structured audit of 21 representative systems, we diagnose a recurring and structurally connected failure pattern: objective collapse, in which single-proxy targets replace multi-objective scientific aims; validation collapse, in which internal self-evaluation replaces independent validation; and acceptance collapse, in which benchmark scores or publication-shaped artifacts replace mechanisms for domain-level critique, reuse, and integration. These collapses are not inherent limits of autonomy but correctable design choices. Accordingly, we outline potential remedies across objective signal, validation, and output pathway to spark community discussion.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 70 of its 81 references that are in the map. First posted on arXiv 25 May 2026.
Builds on
70 of 81 references.
References · 70
- 2025Accelerating scientific discovery with Co-Scientistcited by 353
- 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 289
- 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 252
- 2026Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skillscited by 161
- 2026Towards End-to-End Automation of AI Researchcited by 150
- 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
- 2018Categorizing Variants of Goodhart's Lawcited by 132
- 2025Deep Research Agents: A Systematic Examination And Roadmapcited by 122
- 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forwardcited by 105
- 2026AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolutioncited by 105
- 2026EvoSkill: Automated Skill Discovery for Multi-Agent Systemscited by 96
- 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 74
38 more
- 2026SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?cited by 72
- 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 71
- 2024Feedback Loops With Language Models Drive In-Context Reward Hackingcited by 70
- 2025SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experiencecited by 68
- 2026Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scalecited by 65
- 2025AgentRxiv: Towards Collaborative Autonomous Researchcited by 64
- 2024CRISPR-GPT for Agentic Automation of Gene-editing Experimentscited by 56
- 2026Memento-Skills: Let Agents Design Agentscited by 56
- 2025An AI system to help scientists write expert-level empirical softwarecited by 52
- 2026SkillNet: Create, Evaluate, and Connect AI Skillscited by 48
- 2026CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discoverycited by 38
- 2023Goodhart's Law in Reinforcement Learningcited by 37
- 2026When Single-Agent with Skills Replace Multi-Agent Systems and When They Failcited by 34
- 2026MiroThinker-1.7 & H1: Towards Heavy-Duty Research Agents via Verificationcited by 28
- 2026AutoResearchClaw: Self-Reinforcing Autonomous Research with Human-AI Collaborationcited by 23
- 2025SkillFlow: Scalable and Efficient Agent Skill Retrieval Systemcited by 20
- 2026AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discoverycited by 17
- 2026Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMscited by 13
- 2026Omni-SimpleMem: Autoresearch-Guided Discovery of Lifelong Multimodal Agent Memorycited by 13
- 2026AI scientists produce results without reasoning scientificallycited by 13
- 2026Can LLMs Beat Classical Hyperparameter Optimization Algorithms? A Study on autoresearchcited by 12
- 2026Marco DeepResearch: Unlocking Efficient Deep Research Agents via Verification-Centric Designcited by 12
- 2026EvolveMem:Self-Evolving Memory Architecture via AutoResearch for LLM Agentscited by 11
- 2024Agent Skill Acquisition for Large Language Models via CycleQDcited by 10
- 2026Evolving Medical Imaging Agents via Experience-driven Self-skill Discoverycited by 10
- 2026Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?cited by 10
- 2026How Far Are We From True Auto-Research?cited by 8
- 2026Semi-Autonomous Formalization of the Vlasov-Maxwell-Landau Equilibriumcited by 7
- 2025Foundation Models for Scientific Discovery: From Paradigm Enhancement to Paradigm Transitioncited by 6
- 2026ABSTRAL: Automatic Design of Multi-Agent Systems Through Iterative Refinement and Topology Optimizationcited by 6
- 2026Bilevel Autoresearch: Meta-Autoresearching Itselfcited by 6
- 2026Memory Intelligence Agentcited by 6
- 2026GRAFT-ATHENA: Self-Improving Agentic Teams for Autonomous Discovery and Evolutionary Numerical Algorithmscited by 6
- 2026FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamicscited by 6
- 2026TARSE: Test-Time Adaptation via Retrieval of Skills and Experience for Reasoning Agentscited by 4
- 2026SciIntegrity-Bench: A Benchmark for Evaluating Academic Integrity in AI Scientist Systemscited by 4
- 2026MedMASLab: A Unified Orchestration Framework for Benchmarking Multimodal Medical Multi-Agent Systemscited by 3
- 2026CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpadcited by 3
50 most cited of 70 All 70 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.