Can AI Scientists Coordinate at Runtime?
Not on the map. Why?
Abstract (arXiv)
Multi-agent AI scientists have shown improving performance across a diverse range of tasks. Yet a common approach is design-time agentic orchestration, which typically relies on fixed workflows. In contrast, human scientists coordinate and adjust their division of labor at runtime. We therefore ask: can AI scientists also coordinate at runtime? To this end, we introduce Runtime Agent Coordination (RAC), which selects agents from existing AI-scientist hosts during execution, assigns scoped work contracts, and provides artifact-grounded verification. Verification informs subsequent agents without blocking transitions or discarding artifacts. We conduct a single-seed exploratory evaluation across Agent Laboratory, EvoScientist, and ARK on ResearchClawBench, preserving host models, tools, and permissions under host-calibrated budgets. Four cumulative conditions separate native execution, runtime communication, runtime selection, and the combined addition of contracts and verification. Runtime selection yields the highest observed mean score for each host; adding contracts and verification reduces these means, with host-dependent outcomes relative to native execution. These results motivate runtime coordination while exposing the limits of additional coordination mechanisms under constrained budgets. Code is available at https://github.com/systemind-team/Runtime-AI-Scientist.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 24 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
21 of 24 references.
References · 21
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
- 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
- 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 375
- 2025Accelerating scientific discovery with Co-Scientistcited by 353
- 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaborationcited by 281
- 2024AI Agents That Mattercited by 199
- 2024DiscoveryBench: Towards Data-Driven Discovery with Large Language Modelscited by 97
- 2025AI-Researcher: Autonomous Scientific Innovationcited by 88
9 more
- 2026EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discoverycited by 41
- 2025Training AI Co-Scientists Using Rubric Rewardscited by 23
- 2026AutoResearchClaw: Self-Reinforcing Autonomous Research with Human-AI Collaborationcited by 23
- 2026ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Researchcited by 13
- 2026ARIS: Autonomous Research via Adversarial Multi-Agent Collaborationcited by 12
- 2025Build Your Personalized Research Group: A Multiagent Framework for Continual and Interactive Science Automationcited by 8
- 2026AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentationcited by 7
- 2026How Do Agents Fail on AutoResearch: End-to-End Diagnostic Evaluation on 100 Real-World Frontier Research Taskscited by 4
- 2026Can AI Evaluate AI Scientists? A Benchmarking Study of Autonomous Research Generation Systems Using Automated Multi-Model Reviewcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026When Agents Coordinate: Measuring Coordination in Multi-Agent AI Coding
- 2026Argus: A General-Purpose Agentic Reasoning Runtime for Long-Horizon Tasks
- 2026Governed Evolution of Agent Runtimes through Executable Operational Cognition
- 2026AgentTime: Can Agents Estimate and Control Their Own Runtime?
- 2026A Methodology for Selecting and Composing Runtime Architecture Patterns for Production LLM Agents
- 2025Orchestrating Human-AI Teams: The Manager Agent as a Unifying Research Challenge
- 2026Runtime Governance for AI Agents: Policies on Paths
- 2026AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration