ARIS: Autonomous Research via Adversarial Multi-Agent Collaboration
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
This report describes ARIS (Auto-Research-in-sleep), an open-source research harness for autonomous research, including its architecture, assurance mechanisms, and early deployment experience. The performance of agent systems built on LLMs depends on both the model weights and the harness around them, which governs what information to store, retrieve, and present to the model. For long-horizon research workflows, the central failure mode is not a visible breakdown but a plausible unsupported success: a long-running agent can produce claims whose evidential support is incomplete, misreported, or silently inherited from the executor's framing. Therefore, we present ARIS as a research harness that coordinates machine-learning research workflows through cross-model adversarial collaboration as a default configuration: an executor model drives forward progress while a reviewer from a different model family is recommended to critique intermediate artifacts and request revisions. ARIS has three architectural layers. The execution layer provides more than 65 reusable Markdown-defined skills, model integrations via MCP, a persistent research wiki for iterative reuse of prior findings, and deterministic figure generation. The orchestration layer coordinates five end-to-end workflows with adjustable effort settings and configurable routing to reviewer models. The assurance layer includes a three-stage process for checking whether experimental claims are supported by evidence: integrity verification, result-to-claim mapping, and claim auditing that cross-checks manuscript statements against the claim ledger and raw evidence, as well as a five-pass scientific-editing pipeline, mathematical-proof checks, and visual inspection of the rendered PDF. A prototype self-improvement loop records research traces and proposes harness improvements that are adopted only after reviewer approval.
Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 26 of its 32 references that are in the map. First posted on arXiv 4 May 2026.
Builds on
26 of 32 references.
Closest ancestors · 3
Same direction, newest first
- 2026EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery9 Mar
- 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search10 Apr
- 2025Agent Laboratory: Using LLM Agents as Research Assistants8 Jan
Other references · 23
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
- 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
- 2023ChatDev: Communicative Agents for Software Developmentcited by 919
11 more
- 2024Self-Rewarding Language Modelscited by 681
- 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
- 2025Accelerating scientific discovery with Co-Scientistcited by 401
- 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 309
- 2023Can large language models provide useful feedback on research papers? A large-scale empirical analysiscited by 200
- 2024RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthinesscited by 131
- 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 122
- 2024Autonomous LLM-driven research from data to human-verifiable research paperscited by 53
- 2025The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systemscited by 29
- 2026AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discoverycited by 23
- 2026AI Can Learn Scientific Tastecited by 11
Built on by
Citing papers per year: 2026: 16.
16 papers.
Sort:
By area:
- 2026ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Researchcited by 17
- 2026AutoSci: A Memory-Centric Agentic System for the Full Scientific Research Lifecyclecited by 5
- 2026ResearchLoop: An Evidence-Gated Control Plane for AI-Assisted Researchcited by 3
- 2026Parametric Skillscited by 3
- 2026Towards Autonomous and Auditable Medical Imaging Model Developmentcited by 2
- 2026VALG: An Agentic System for ML Theory Researchcited by 2
- 2026Science Earth: Towards A Planet-Scale Operating System for AI-Native Scientific Discoverycited by 1
- 2026Making AI Scientists Auditable from Evidence to Claimcited by 1
- 2026One Reflection Is Not Enough: Self-Correcting Autonomous Research via Multi-Hypothesis Failure Attributioncited by 1
- 2026ReasFlow: Assisting Reasoning-Centric Scientific Discovery in Applied Mathematics via a Knowledge-Based Multi-Agent Systemcited by 1
- 2026ARAC: Benchmarking Auto-Research's Alignment and Completeness on End-to-End Researchscited by 1
- 2026Clarus: Coordinating Autonomous Research Agents toward Web-Scale Scientific Collaborationcited by 0
4 more
- 2026Towards a new paradigm of scientific discovery with socialized artificial intelligencecited by 0
- 2026Can AI Scientists Coordinate at Runtime?cited by 0
- 2026JANUS: Enhancing Long-Horizon Autonomous Research with Co-Evolving World Modelscited by 0
- 2026ResearchFlow-Eval: Tracing the Execution Gap in End-to-End Multi-Claim Autoresearchcited by 0
- 2026ResearchFlow-Eval: Tracing the Execution Gap in End-to-End Multi-Claim Autoresearch13 Oct 2026
- 2026JANUS: Enhancing Long-Horizon Autonomous Research with Co-Evolving World Models9 Oct 2026
- 2026Can AI Scientists Coordinate at Runtime?1 Oct 2026
- 2026VALG: An Agentic System for ML Theory Research13 Aug 2026
- 2026ARAC: Benchmarking Auto-Research's Alignment and Completeness on End-to-End Researchs13 Aug 2026
- 2026Towards a new paradigm of scientific discovery with socialized artificial intelligence3 Aug 2026
- 2026ReasFlow: Assisting Reasoning-Centric Scientific Discovery in Applied Mathematics via a Knowledge-Based Multi-Agent System15 Jul 2026
- 2026Towards Autonomous and Auditable Medical Imaging Model Development12 Jul 2026
- 2026One Reflection Is Not Enough: Self-Correcting Autonomous Research via Multi-Hypothesis Failure Attribution30 Jun 2026
- 2026Clarus: Coordinating Autonomous Research Agents toward Web-Scale Scientific Collaboration29 Jun 2026
- 2026Parametric Skills29 Jun 2026
- 2026Making AI Scientists Auditable from Evidence to Claim17 Jun 2026
4 more
- 2026Science Earth: Towards A Planet-Scale Operating System for AI-Native Scientific Discovery31 May 2026
- 2026AutoSci: A Memory-Centric Agentic System for the Full Scientific Research Lifecycle29 May 2026
- 2026ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research28 May 2026
- 2026ResearchLoop: An Evidence-Gated Control Plane for AI-Assisted Research27 May 2026
- 2026ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Researchcited by 17
- 2026AutoSci: A Memory-Centric Agentic System for the Full Scientific Research Lifecyclecited by 5
- 2026VALG: An Agentic System for ML Theory Researchcited by 2
- 2026One Reflection Is Not Enough: Self-Correcting Autonomous Research via Multi-Hypothesis Failure Attributioncited by 1
- 2026Clarus: Coordinating Autonomous Research Agents toward Web-Scale Scientific Collaborationcited by 0
- 2026VALG: An Agentic System for ML Theory Research13 Aug 2026
- 2026One Reflection Is Not Enough: Self-Correcting Autonomous Research via Multi-Hypothesis Failure Attribution30 Jun 2026
- 2026Clarus: Coordinating Autonomous Research Agents toward Web-Scale Scientific Collaboration29 Jun 2026
- 2026AutoSci: A Memory-Centric Agentic System for the Full Scientific Research Lifecycle29 May 2026
- 2026ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research28 May 2026
- 2026ResearchLoop: An Evidence-Gated Control Plane for AI-Assisted Researchcited by 3
- 2026Parametric Skillscited by 3
- 2026Towards Autonomous and Auditable Medical Imaging Model Developmentcited by 2
- 2026Science Earth: Towards A Planet-Scale Operating System for AI-Native Scientific Discoverycited by 1
- 2026Making AI Scientists Auditable from Evidence to Claimcited by 1
- 2026ARAC: Benchmarking Auto-Research's Alignment and Completeness on End-to-End Researchscited by 1
- 2026Towards a new paradigm of scientific discovery with socialized artificial intelligencecited by 0
- 2026Can AI Scientists Coordinate at Runtime?cited by 0
- 2026ResearchFlow-Eval: Tracing the Execution Gap in End-to-End Multi-Claim Autoresearchcited by 0
- 2026ResearchFlow-Eval: Tracing the Execution Gap in End-to-End Multi-Claim Autoresearch13 Oct 2026
- 2026Can AI Scientists Coordinate at Runtime?1 Oct 2026
- 2026ARAC: Benchmarking Auto-Research's Alignment and Completeness on End-to-End Researchs13 Aug 2026
- 2026Towards a new paradigm of scientific discovery with socialized artificial intelligence3 Aug 2026
- 2026Towards Autonomous and Auditable Medical Imaging Model Development12 Jul 2026
- 2026Parametric Skills29 Jun 2026
- 2026Making AI Scientists Auditable from Evidence to Claim17 Jun 2026
- 2026Science Earth: Towards A Planet-Scale Operating System for AI-Native Scientific Discovery31 May 2026
- 2026ResearchLoop: An Evidence-Gated Control Plane for AI-Assisted Research27 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026FARS: A Fully Automated Research System Deployed at Scale
- 2026Toward Autonomous Long-Horizon Engineering for ML Research
- 2026AlphaLab: Autonomous Multi-Agent Research Across Optimization Domains with Frontier LLMs
- 2026DeltaML-Bench: Evaluating Machine Learning Agents on Real-World Research Repositories
- 2025ADRA-Bank: A Modular Benchmark for Academic Deep Research Agents
- 2026ML-AutoResearch: Training Machine Learning Research Agents with Automatically Generated Environments
- 2026AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents
- 2026AgonAlpha: Autonomous Alpha Discovery via Prompt Economy and Scalable Agentic Search