# SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents

arXiv: https://arxiv.org/abs/2602.12984 · HTML: https://mlsanity.com/paper/2602.12984/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 26 of its 38 references that are in the map.
- First posted on arXiv 13 Feb 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 26 of 26)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark](https://mlsanity.com/paper/2409.02813/) (2024)
- [GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models](https://mlsanity.com/paper/2508.06471/) (2025)
- [Search-o1: Agentic Search-Enhanced Large Reasoning Models](https://mlsanity.com/paper/2501.05366/) (2025)
- [SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models](https://mlsanity.com/paper/2307.10635/) (2023)
- [On the Tool Manipulation Capability of Open-source Large Language Models](https://mlsanity.com/paper/2305.16504/) (2023)
- [DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents](https://mlsanity.com/paper/2406.06769/) (2024)
- [From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery](https://mlsanity.com/paper/2508.14111/) (2025)
- [AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2509.08755/) (2025)
- [SciAgent: Tool-augmented Language Models for Scientific Reasoning](https://mlsanity.com/paper/2402.11451/) (2024)
- [SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?](https://mlsanity.com/paper/2507.05241/) (2025)
- [SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models](https://mlsanity.com/paper/2401.07950/) (2024)
- [R-Bench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluation](https://mlsanity.com/paper/2505.02018/) (2025)
- [Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoning](https://mlsanity.com/paper/2506.10521/) (2025)
- [debug-gym: A Text-Based Environment for Interactive Debugging](https://mlsanity.com/paper/2503.21557/) (2025)
- [RBench-V: A Primary Assessment for Visual Reasoning Models with Multi-modal Outputs](https://mlsanity.com/paper/2505.16770/) (2025)
- [BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset](https://mlsanity.com/paper/2507.03483/) (2025)
- [A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Tools](https://mlsanity.com/paper/2506.20743/) (2025)

## Built on by (most cited first, 15 of 15)

- [Benchmarking AI Agents for Addressing Scientific Challenges Across Scales](https://mlsanity.com/paper/2606.12736/) (2026)
- [HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark](https://mlsanity.com/paper/2604.13954/) (2026)
- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [6GAgentGym: Tool Use, Data Synthesis, and Agentic Learning for Network Management](https://mlsanity.com/paper/2603.29656/) (2026)
- [AgentTrails: Towards Trust and Reuse for Agentic Tasks](https://mlsanity.com/paper/2607.18816/) (2026)
- [AlphaEval: Evaluating Agents in Production](https://mlsanity.com/paper/2604.12162/) (2026)
- [FrontierChallenge: Evaluating Scientific Workflow Completion](https://mlsanity.com/paper/2608.24979/) (2026)
- [Process-Reward Tactic Evolution for Long-Horizon Bioinformatics Workflows](https://mlsanity.com/paper/2606.20839/) (2026)
- [Molecular LLM Agents: From Architectural Design to Scientific Autonomy](https://mlsanity.com/paper/2608.23104/) (2026)
- [EarthVerse: Benchmarking Scientific Agents Across Dynamic Earth Systems and Natural Hazards](https://mlsanity.com/paper/2608.23525/) (2026)
- [WhatWorkedBench: Benchmarking Experimental Understanding in AI Agents](https://mlsanity.com/paper/2609.27490/) (2026)
- [The Marathon of Scientific Reasoning: Robustness of Scientific Agents to Perturbations in Multi-Turn Interactions](https://mlsanity.com/paper/2609.34537/) (2026)
- [Agents Are Systems, Not Models: Rethinking Agentic Evaluation](https://mlsanity.com/paper/2610.01618/) (2026)
- [NeutronGym: Physics-Graded Neutron Instrument Design for LLM Agents](https://mlsanity.com/paper/2610.03631/) (2026)
- [ScienceClaw: Benchmarking Continual Self-Evolution of AI-for-Science Agents Across the Natural and Social Sciences](https://mlsanity.com/paper/2610.08691/) (2026)

## Abstract (arXiv)

Scientific reasoning inherently demands integrating sophisticated toolkits to navigate domain-specific knowledge. Yet, current benchmarks largely overlook agents' ability to orchestrate tools for such rigorous workflows. To bridge this gap, we introduce SciAgentGym, a scalable interactive environment featuring 1,780 domain-specific tools across four natural science disciplines, supported by a robust execution infrastructure. Complementing this, we present SciAgentBench, a tiered evaluation suite designed to stress-test agentic capabilities from elementary actions to long-horizon workflows. Our evaluation identifies a critical bottleneck: state-of-the-art models still struggle with complex scientific tool-use, and their performance degrades substantially as interaction horizons extend. To address this, we propose SciForge, a data synthesis method that models the tool action space as a dependency graph to generate logic-aware training trajectories. By fine-tuning on these trajectories, our SciAgent-8B outperforms the significantly larger Qwen3-VL-235B-Instruct while exhibiting positive cross-domain transfer of scientific tool-use capabilities. These results underscore the promising potential of next-generation autonomous scientific agents.