mlsanityData to 14 Oct 2026

From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery

arXiv 2505.13259 ↗First posted 19 May 2025cs.CLcited by 93

Not on the map. Why?

Cited by 32 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large Language Models (LLMs) are catalyzing a paradigm shift in scientific discovery, evolving from task-specific automation tools into increasingly autonomous agents and fundamentally redefining research processes and human-AI collaboration. This survey systematically charts this burgeoning field, placing a central focus on the changing roles and escalating capabilities of LLMs in science. Through the lens of the scientific method, we introduce a foundational three-level taxonomy-Tool, Analyst, and Scientist-to delineate their escalating autonomy and evolving responsibilities within the research lifecycle. We further identify pivotal challenges and future research trajectories such as robotic automation, self-improvement, and ethical governance. Overall, this survey provides a conceptual architecture and strategic foresight to navigate and shape the future of AI-driven scientific discovery, fostering both rapid innovation and responsible advancement. Github Repository: https://github.com/HKUST-KnowComp/Awesome-LLM-Scientific-Discovery.

Key facts. Cited by 93 papers in the map (data through 14 Oct 2026). Builds on 80 of its 117 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

80 of 117 references.

References · 80

  1. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  2. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  3. 2023Teaching Large Language Models to Self-Debugcited by 988
  4. 2024A Survey on Large Language Models for Code Generationcited by 658
  5. 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 548
  6. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 469
  7. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 452
  8. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 425
  9. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 422
  10. 2024Understanding the planning of LLM agents: A surveycited by 421
  11. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 366
  12. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
38 more
  1. 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 234
  2. 2025AIDE: AI-Driven Exploration in the Space of Codecited by 226
  3. 2024Tool Learning with Large Language Models: A Surveycited by 223
  4. 2022End-to-end symbolic regression with transformerscited by 203
  5. 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 188
  6. 2023SciMON: Scientific Inspiration Machines Optimized for Noveltycited by 179
  7. 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 175
  8. 2023PaperQA: Retrieval-Augmented Generative Agent for Scientific Researchcited by 149
  9. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 148
  10. 2024InfiAgent-DABench: Evaluating Agents on Data Analysis Taskscited by 140
  11. 2024ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoningcited by 128
  12. 2024DSBench: How Far Are Data Science Agents from Becoming Data Science Experts?cited by 125
  13. 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 122
  14. 2024AutoSurvey: Large Language Models Can Automatically Write Surveyscited by 119
  15. 2024Empowering Biomedical Discovery with AI Agentscited by 118
  16. 2023Large Language Models for Automated Open-domain Scientific Hypotheses Discoverycited by 100
  17. 2024DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agentscited by 100
  18. 2024Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agentscited by 97
  19. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  20. 2023CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalizationcited by 95
  21. 2024SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoningcited by 94
  22. 2022Natural Language to Code Generation in Interactive Data Science Notebookscited by 87
  23. 2023AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZcited by 79
  24. 2024BLADE: Benchmarking Language Model Agents for Data-Driven Sciencecited by 77
  25. 2025Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?cited by 75
  26. 2024MLR-Copilot: Autonomous Machine Learning Research based on Large Language Models Agentscited by 67
  27. 2025ResearchBench: Benchmarking LLMs in Scientific Discovery via Inspiration-Based Task Decompositioncited by 66
  28. 2024LitLLM: A Toolkit for Scientific Literature Reviewcited by 63
  29. 2018Automated Machine Learning: From Principles to Practicescited by 62
  30. 2024Nova: An Iterative Planning and Search Approach to Enhance Novelty and Diversity of LLM Generated Ideascited by 57
  31. 2025CodeScientist: End-to-End Semi-Automated Scientific Discovery with Code-based Experimentationcited by 57
  32. 2024AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviewscited by 44
  33. 2024DrugAgent: Automating AI-aided Drug Discovery Programming through LLM Multi-Agent Collaborationcited by 44
  34. 2023BioPlanner: Automatic Evaluation of LLMs on Protocol Planning in Biologycited by 43
  35. 2025SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Paperscited by 43
  36. 2024Automated Statistical Model Discovery with Language Modelscited by 41
  37. 2024Human-LLM Compound System for Scientific Ideation through Facet Recombination and Novelty Evaluationcited by 41
  38. 2024Tables as Texts or Images: Evaluating the Table Reasoning Ability of LLMs and MLLMscited by 40

50 most cited of 80 All 80 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 23; 2026: 70.

93 papers.

Sort:

By area:

  1. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 77
  2. 2026Towards Autonomous Mathematics Researchcited by 40
  3. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 37
  4. 2025NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agentscited by 36
  5. 2026Toward Ultra-Long-Horizon Agentic Science: Cognitive Accumulation for Machine Learning Engineeringcited by 28
  6. 2025AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corporacited by 23
  7. 2025How Far Are AI Scientists from Changing the World?cited by 15
  8. 2025Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physicscited by 11
  9. 2026AI for Auto-Research: Roadmap & User Guidecited by 11
  10. 2025Synthelite: Chemist-aligned and feasibility-aware synthesis planning with LLMscited by 10
  11. 2026SciCoQA: Quality Assurance for Scientific Paper--Code Alignmentcited by 10
  12. 2026AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discoverycited by 10
38 more
  1. 2026AI Research Preference Modelscited by 10
  2. 2025Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluatorcited by 9
  3. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 9
  4. 2025LLM-Hanabi: Evaluating Multi-Agent Gameplays with Theory-of-Mind and Rationale Inference in Imperfect Information Collaboration Gamecited by 8
  5. 2025AutoGraph-R1: End-to-End Reinforcement Learning for Knowledge Graph Constructioncited by 7
  6. 2025Deep Ideation: Designing LLM Agents to Generate Novel Research Ideas on Scientific Concept Networkcited by 7
  7. 2026Principle-Evolvable Scientific Discovery via Uncertainty Minimizationcited by 7
  8. 2025ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoningcited by 6
  9. 2026Effective Harness Engineering for Algorithm Discovery with Coding Agentscited by 6
  10. 2026Camyla: Scaling Autonomous Research in Medical Image Segmentationcited by 5
  11. 2026SciIntegrity-Bench: A Benchmark for Evaluating Academic Integrity in AI Scientist Systemscited by 5
  12. 2025Rethinking Prospect Theory for LLMs: Revealing the Instability of Decision-Making under Epistemic Uncertaintycited by 4
  13. 2025DixitWorld: Evaluating Multimodal Abductive Reasoning in Vision-Language Models with Multi-Agent Dixit Gameplaycited by 4
  14. 2025CritiCal: Can Critique Help LLM Uncertainty or Confidence Calibration?cited by 4
  15. 2025DataGovBench: Benchmarking LLM Agents for Real-World Data Governance Workflowscited by 4
  16. 2025VASP Agent: An Agentic Framework for Autonomous First-principles Calculationscited by 4
  17. 2026Sci-Reasoning: A Dataset Decoding AI Innovation Patternscited by 4
  18. 2026HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learningcited by 4
  19. 2026The Agentic Researcher: A Practical Guide to AI-Assisted Research in Mathematics and Machine Learningcited by 4
  20. 2025GenCellAgent: Generalizable, Training-Free Cellular Image Segmentation via Large Language Model Agentscited by 3
  21. 2026CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpadcited by 3
  22. 2026GTA-2: Benchmarking General Tool Agents from Atomic Tool-Use to Open-Ended Workflowscited by 3
  23. 2026EvoSci: A Bio-Inspired Multi-Agent Framework for the Evolution of Scientific Discoverycited by 3
  24. 2025Structuring the Unstructured: A Systematic Review of Text-to-Structure Generation for Agentic AI with a Universal Evaluation Frameworkcited by 2
  25. 2025AutoPR: Let's Automate Your Academic Promotion!cited by 2
  26. 2025Knowledge-Informed Automatic Feature Extraction via Collaborative Large Language Model Agentscited by 2
  27. 2026MolQuest: A Benchmark for Agentic Evaluation of Abductive Reasoning in Chemical Structure Elucidationcited by 2
  28. 2026RELISH: LLM REgression with a Latent Iterative State Headcited by 2
  29. 2026MaD Physics: Evaluating information seeking under constraints in physical environmentscited by 2
  30. 2026STRIDE: A Self-Reflective Agent Framework for Reliable Automatic Equation Discoverycited by 2
  31. 2026ResearchStudio-Idea: An Evidence-Grounded Research-Ideation Skill Suite from ML Conference Outcomescited by 2
  32. 2026Distributed Denial of Science: How Indirect Data Poisoning of AI Systems Can Industrialize Scientific Fraudcited by 2
  33. 2026Can Large Language Models Forecast What Researchers Study Next?cited by 2
  34. 2025HypoChainer: A Collaborative System Combining LLMs and Knowledge Graphs for Hypothesis-Driven Scientific Discoverycited by 1
  35. 2025Speak to a Protein: An Interactive Multimodal Co-Scientistcited by 1
  36. 2026AutoMS: Multi-Agent Evolutionary Search for Cross-Physics Inverse Microstructure Designcited by 1
  37. 2026MedConclusion: A Benchmark for Biomedical Conclusion Generation from Structured Abstractscited by 1
  38. 2026An AI Co-Data-Scientist for Prioritizing Candidate Biomarkers from Wearable Sensor Datacited by 1

50 most cited of 93 All 93 (JSON)