# Benchmarking AI Agents for Addressing Scientific Challenges Across Scales

arXiv: https://arxiv.org/abs/2606.12736 · HTML: https://mlsanity.com/paper/2606.12736/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 145 references that are in the map.
- First posted on arXiv 10 Jun 2026.

## Builds on (most cited first, 22 of 22)

- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026)
- [Accelerating scientific discovery with Co-Scientist](https://mlsanity.com/paper/2502.18864/) (2025)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [MLGym: A New Framework and Benchmark for Advancing AI Research Agents](https://mlsanity.com/paper/2502.14499/) (2025)
- [BixBench: a Comprehensive Benchmark for LLM-based Agents in Computational Biology](https://mlsanity.com/paper/2503.00096/) (2025)
- [Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2503.08979/) (2025)
- [Deep Research: A Survey of Autonomous Research Agents](https://mlsanity.com/paper/2508.12752/) (2025)
- [MedAgentBench: A Realistic Virtual EHR Environment to Benchmark Medical LLM Agents](https://mlsanity.com/paper/2501.14654/) (2025)
- [TxAgent: An AI Agent for Therapeutic Reasoning Across a Universe of Tools](https://mlsanity.com/paper/2503.10970/) (2025)
- [AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite](https://mlsanity.com/paper/2510.21652/) (2025)
- [DrugAgent: Automating AI-aided Drug Discovery Programming through LLM Multi-Agent Collaboration](https://mlsanity.com/paper/2411.15692/) (2024)
- [Evaluating Large Language Models in Scientific Discovery](https://mlsanity.com/paper/2512.15567/) (2025)
- [ToolUniverse: An open platform for democratizing AI scientists](https://mlsanity.com/paper/2509.23426/) (2025)
- [SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Tasks](https://mlsanity.com/paper/2507.01001/) (2025)
- [Accelerating Scientific Discovery with Autonomous Goal-evolving Agents](https://mlsanity.com/paper/2512.21782/) (2025)
- [SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents](https://mlsanity.com/paper/2602.12984/) (2026)
- [LitLLMs, LLMs for Literature Review: Are we there yet?](https://mlsanity.com/paper/2412.15249/) (2024)
- [CellForge: Agentic Design of Virtual Cell Models](https://mlsanity.com/paper/2508.02276/) (2025)
- [BioAgent Bench: An AI Agent Evaluation Suite for Bioinformatics](https://mlsanity.com/paper/2601.21800/) (2026)
- [Benchmarking AI scientists for omics data driven biological discovery](https://mlsanity.com/paper/2505.08341/) (2025)
- [DSAEval: Evaluating Data Science Agents on a Wide Range of Real-World Data Science Problems](https://mlsanity.com/paper/2601.13591/) (2026)

## Built on by (most cited first, 12 of 12)

- [ASI-Bench: At the Dawn of Artificial Superintelligence](https://mlsanity.com/paper/2608.17271/) (2026)
- [FrontierChallenge: Evaluating Scientific Workflow Completion](https://mlsanity.com/paper/2608.24979/) (2026)
- [Molecular LLM Agents: From Architectural Design to Scientific Autonomy](https://mlsanity.com/paper/2608.23104/) (2026)
- [AGENT-O: A Semantic Agent Card Framework for Interoperable and Governed Healthcare AI Agents](https://mlsanity.com/paper/2608.28345/) (2026)
- [ToolGate: An Executable Acceptance Pipeline for Tool-Dependent Scientific Benchmark Construction](https://mlsanity.com/paper/2609.02067/) (2026)
- [WhatWorkedBench: Benchmarking Experimental Understanding in AI Agents](https://mlsanity.com/paper/2609.27490/) (2026)
- [From Dead Code and Static Requirements to Working Engines: Software Revival with Coding Agents](https://mlsanity.com/paper/2609.36161/) (2026)
- [OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software](https://mlsanity.com/paper/2609.39903/) (2026)
- [EurekaBench: Measuring Agentic Ability to Discover New Scientific Insights](https://mlsanity.com/paper/2610.00492/) (2026)
- [MedicalHarness: A Controlled Evaluation of LLMs and Agent Harnesses on Medical Tasks](https://mlsanity.com/paper/2610.05778/) (2026)
- [ScienceClaw: Benchmarking Continual Self-Evolution of AI-for-Science Agents Across the Natural and Social Sciences](https://mlsanity.com/paper/2610.08691/) (2026)
- [OpenProblemBench: Benchmarking AI on Open Problems in the Foundational Theoretical Sciences](https://mlsanity.com/paper/2610.11118/) (2026)

## Abstract (arXiv)

AI agents are increasingly being developed to accelerate scientific discovery, yet their practical capabilities in real research settings remain poorly understood. Existing benchmarks for AI agents rarely capture the complexity, heterogeneity, and extended reasoning required by scientific work, whereas benchmarks for scientific tasks often reduce research to static, direct problems and provide limited support for interactive evaluation. Here, we introduce SciAgentArena, a systematic benchmark for evaluating AI agents in real-world scientific research scenarios drawn from emerging needs across multiple domains. SciAgentArena comprises approximately 200 tasks with stepwise verification and an interactive, agent-agnostic environment for assessing diverse AI agents. Using this benchmark, we find that current agents can contribute effectively to well-specified data-analysis workflows, particularly when the task structure and evaluation criteria are clear. However, their performance remains uneven across scientific contexts: agents struggle to generate genuinely novel insights, sustain self-directed exploration, and formulate robust solutions for open-ended research questions. We further characterize common failure modes across agents and identify opportunities for improving their reliability, autonomy, and scientific reasoning. Together, SciAgentArena provides a practical framework for measuring progress in AI agents for science and for guiding the design of future agents capable of addressing complex scientific challenges. Full codes, tasks, and datasets can be accessed via this link: https://sciagentarena.github.io/.