mlsanityData to 14 Oct 2026

Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents

arXiv 2503.24047 ↗First posted 31 Mar 2025cs.AI cs.MAcited by 98

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

As scientific research becomes increasingly complex, innovative tools are needed to manage vast data, facilitate interdisciplinary collaboration, and accelerate discovery. Large language models (LLMs) are now evolving into LLM-based scientific agents that automate critical tasks ranging from hypothesis generation and experiment design to data analysis and simulation. Unlike general-purpose LLMs, these specialized agents integrate domain-specific knowledge, advanced tool sets, and robust validation mechanisms, enabling them to handle complex data types, ensure reproducibility, and drive scientific breakthroughs. This survey provides a focused review of the architectures, design, benchmarks, applications, and ethical considerations surrounding LLM-based scientific agents. We highlight why they differ from general agents and the ways in which they advance research across various scientific fields. By examining their development and challenges, this survey offers a comprehensive roadmap for researchers and practitioners to harness these agents for more efficient, reliable, and ethically sound scientific discovery.

Key facts. Cited by 98 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 87 of its 128 references that are in the map. First posted on arXiv 31 Mar 2025.

Builds on

87 of 128 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025PaSa: An LLM Agent for Comprehensive Academic Paper Search17 Jan
  2. 2025Agent Laboratory: Using LLM Agents as Research Assistants8 Jan
  3. 2024Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agents17 Oct
  4. 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery7 Oct
  5. 2024SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning9 Sep
  6. 2024MLR-Copilot: Autonomous Machine Learning Research based on Large Language Models Agents26 Aug
  7. 2024ChatCite: LLM Agent with Human Workflow Guidance for Comparative Literature Summary5 Mar
  8. 2024SciAgent: Tool-augmented Language Models for Scientific Reasoning18 Feb
  9. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation5 Oct

Other references · 78

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  6. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  7. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  8. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  9. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  10. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  11. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  12. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
38 more
  1. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
  2. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  3. 2025Humanity's Last Examcited by 491
  4. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 441
  5. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  6. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 383
  7. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 319
  8. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 316
  9. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 273
  10. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 255
  11. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 239
  12. 2024V-STaR: Training Verifiers for Self-Taught Reasonerscited by 217
  13. 2023SciMON: Scientific Inspiration Machines Optimized for Noveltycited by 179
  14. 2024SciCode: A Research Coding Benchmark Curated by Scientistscited by 178
  15. 2023A Study of Generative Large Language Model for Medical Research and Healthcarecited by 163
  16. 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 156
  17. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 144
  18. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 132
  19. 2024AgentScope: A Flexible yet Robust Multi-Agent Platformcited by 121
  20. 2024Empowering Biomedical Discovery with AI Agentscited by 118
  21. 2023GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Informationcited by 117
  22. 2023SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Researchcited by 115
  23. 2024A Survey on Large Language Model-Based Game Agentscited by 112
  24. 2024DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agentscited by 100
  25. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 99
  26. 202314 Examples of How LLMs Can Transform Materials Science and Chemistry: A Reflection on a Large Language Model Hackathoncited by 86
  27. 2024LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discoverycited by 86
  28. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 84
  29. 2025Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?cited by 75
  30. 2023MechAgents: Large language model multi-agent collaborations can solve mechanics problems, generate new data, and integrate knowledgecited by 63
  31. 2023DrugAssist: A Large Language Model for Molecule Optimizationcited by 59
  32. 2024HoneyComb: A Flexible LLM-Based Agent System for Materials Sciencecited by 57
  33. 2024ProLLaMA: A Protein Large Language Model for Multi-Task Protein Language Processingcited by 53
  34. 2024BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experimentscited by 53
  35. 2021Scientific multi-agent reinforcement learning for wall-models of turbulent flowscited by 52
  36. 2024ClimateGPT: Towards AI Synthesizing Interdisciplinary Research on Climate Changecited by 47
  37. 2024GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solvingcited by 45
  38. 2023BioPlanner: Automatic Evaluation of LLMs on Protocol Planning in Biologycited by 43

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 43; 2026: 55.

98 papers.

Sort:

By area:

  1. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
  2. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  3. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 109
  4. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 77
  5. 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Frameworkcited by 54
  6. 2025TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Usecited by 51
  7. 2026Code as Agent Harnesscited by 47
  8. 2025Agentic Web: Weaving the Next Web with AI Agentscited by 45
  9. 2025Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Livecited by 44
  10. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 44
  11. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 37
  12. 2025DREAMS: Density Functional Theory Based Research Engine for Agentic Materials Simulationcited by 29
38 more
  1. 2025PiFlow: Principle-Aware Scientific Discovery with Multi-Agent Collaborationcited by 28
  2. 2025LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarioscited by 26
  3. 2026Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunitiescited by 20
  4. 2025KTAE: A Model-Free Algorithm to Key-Tokens Advantage Estimation in Mathematical Reasoningcited by 19
  5. 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?cited by 18
  6. 2025BioProBench: A Corpus and Benchmark for Biological Protocol Reasoning in Autonomous Sciencecited by 16
  7. 2025Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language Modelscited by 16
  8. 2025How Far Are AI Scientists from Changing the World?cited by 15
  9. 2025AgenticSciML: Collaborative Multi-Agent Systems for Emergent Discovery in Scientific Machine Learningcited by 14
  10. 2026Agentic Artificial Intelligence (AI): Architectures, Taxonomies, and Evaluation of Large Language Model Agentscited by 14
  11. 2025CellForge: Agentic Design of Virtual Cell Modelscited by 13
  12. 2025Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questionscited by 11
  13. 2025Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physicscited by 11
  14. 2026The Why Behind the Action: Unveiling Internal Drivers via Agentic Attributioncited by 11
  15. 2025Trustless Autonomy: Understanding Motivations, Benefits, and Governance Dilemmas in Self-Sovereign Decentralized AI Agentscited by 10
  16. 2026AI Research Preference Modelscited by 10
  17. 2025ACE-RL: Adaptive Constraint-Enhanced Reward for Long-form Generation Reinforcement Learningcited by 9
  18. 2026ReplicatorBench: Benchmarking LLM Agents for Replicability in Social and Behavioral Sciencescited by 9
  19. 2026PDEAgent-Bench: A Multi-Metric, Multi-Library Benchmark for PDE Solver Generationcited by 8
  20. 2026Tracing Agentic Failure from the Flow of Successcited by 8
  21. 2026HyperSkill: Self-Evolving LLM Agents via Hypergraph-Structured Skill Memorycited by 7
  22. 2025URSA: The Universal Research and Scientific Agentcited by 6
  23. 2025Uncovering Vulnerabilities of LLM-Assisted Cyber Threat Intelligencecited by 6
  24. 2025SCI-Verifier: Scientific Verifier with Thinkingcited by 6
  25. 2025InferA: A Smart Assistant for Cosmological Ensemble Datacited by 6
  26. 2025Foundation Models for Scientific Discovery: From Paradigm Enhancement to Paradigm Transitioncited by 6
  27. 2025Improving MLLM's Document Image Machine Translation via Synchronously Self-reviewing Its OCR Proficiencycited by 5
  28. 2025BioDisco: Multi-agent hypothesis generation with dual-mode evidence, iterative feedback and temporal evaluationcited by 5
  29. 2026SimMOF: AI agent for Automated MOF Simulationscited by 5
  30. 2025Comparing Exploration-Exploitation Strategies of LLMs and Humans: Insights from Standard Multi-armed Bandit Experimentscited by 4
  31. 2025Evolving Excellence: Automated Optimization of LLM-based Agentscited by 4
  32. 2026S1-NexusAgent: a Self-Evolving Agent Framework for Multidisciplinary Scientific Researchcited by 4
  33. 2026AgentWebBench: Benchmarking Multi-Agent Coordination in Agentic Webcited by 4
  34. 2025Towards Fully Automated Molecular Simulations: Multi-Agent Framework for Simulation Setup and Force Field Extractioncited by 3
  35. 2025SimulRAG: Simulator-based RAG for Grounding LLMs in Long-form Scientific QAcited by 3
  36. 2025Sophia: A Persistent Agent Framework of Artificial Lifecited by 3
  37. 2026PaperAudit-Bench: Benchmarking Error Detection in Research Papers for Critical Automated Peer Reviewcited by 3
  38. 2026SimulCost: A Cost-Aware Benchmark and Toolkit for Automating Physics Simulations with LLMscited by 3

50 most cited of 98 All 98 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025LLM-Based Data Science Agents: A Survey of Capabilities, Challenges, and Future Directions
  2. 2025Large Language Model-based Data Science Agent: A Survey
  3. 2024A Survey on Large Language Model-based Agents for Statistics and Data Science
  4. 2024Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System
  5. 2023Large Language Models are Zero Shot Hypothesis Proposers
  6. 2025The Empowerment of Science of Science by Large Language Models: New Tools and Methods
  7. 2025Advancing AI-Scientist Understanding: Multi-Agent LLMs with Interpretable Physics Reasoning
  8. 2025Scientific Hypothesis Generation and Validation: Methods, Datasets, and Future Directions