# From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery

arXiv: https://arxiv.org/abs/2505.13259 · HTML: https://mlsanity.com/paper/2505.13259/

## Key facts

- Cited by 93 papers in the map (data through 14 Oct 2026).
- Builds on 80 of its 117 references that are in the map.
- First posted on arXiv 19 May 2025.

## Builds on (most cited first, 30 of 80)

- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering](https://mlsanity.com/paper/2410.07095/) (2024)
- [DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation](https://mlsanity.com/paper/2211.11501/) (2022)
- [Agent Laboratory: Using LLM Agents as Research Assistants](https://mlsanity.com/paper/2501.04227/) (2025)
- [The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search](https://mlsanity.com/paper/2504.08066/) (2025)
- [Understanding the planning of LLM agents: A survey](https://mlsanity.com/paper/2402.02716/) (2024)
- [MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation](https://mlsanity.com/paper/2310.03302/) (2023)
- [Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers](https://mlsanity.com/paper/2409.04109/) (2024)
- [CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs](https://mlsanity.com/paper/2406.18521/) (2024)
- [AIDE: AI-Driven Exploration in the Space of Code](https://mlsanity.com/paper/2502.13138/) (2025)
- [Tool Learning with Large Language Models: A Survey](https://mlsanity.com/paper/2405.17935/) (2024)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding](https://mlsanity.com/paper/2401.04398/) (2024)
- [SciMON: Scientific Inspiration Machines Optimized for Novelty](https://mlsanity.com/paper/2305.14259/) (2023)
- [RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts](https://mlsanity.com/paper/2411.15114/) (2024)
- [PaperQA: Retrieval-Augmented Generative Agent for Scientific Research](https://mlsanity.com/paper/2312.07559/) (2023)
- [CycleResearcher: Improving Automated Research via Automated Review](https://mlsanity.com/paper/2411.00816/) (2024)
- [InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks](https://mlsanity.com/paper/2401.05507/) (2024)
- [ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning](https://mlsanity.com/paper/2402.12185/) (2024)
- [DSBench: How Far Are Data Science Agents from Becoming Data Science Experts?](https://mlsanity.com/paper/2409.07703/) (2024)
- [ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing](https://mlsanity.com/paper/2306.00622/) (2023)
- [AutoSurvey: Large Language Models Can Automatically Write Surveys](https://mlsanity.com/paper/2406.10252/) (2024)
- [Empowering Biomedical Discovery with AI Agents](https://mlsanity.com/paper/2404.02831/) (2024)
- [Large Language Models for Automated Open-domain Scientific Hypotheses Discovery](https://mlsanity.com/paper/2309.02726/) (2023)
- [DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents](https://mlsanity.com/paper/2406.06769/) (2024)
- [Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agents](https://mlsanity.com/paper/2410.13185/) (2024)

## Built on by (most cited first, 30 of 93)

- [From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery](https://mlsanity.com/paper/2508.14111/) (2025)
- [Towards Autonomous Mathematics Research](https://mlsanity.com/paper/2602.10177/) (2026)
- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents](https://mlsanity.com/paper/2510.07172/) (2025)
- [Toward Ultra-Long-Horizon Agentic Science: Cognitive Accumulation for Machine Learning Engineering](https://mlsanity.com/paper/2601.10402/) (2026)
- [AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora](https://mlsanity.com/paper/2505.23628/) (2025)
- [How Far Are AI Scientists from Changing the World?](https://mlsanity.com/paper/2507.23276/) (2025)
- [Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physics](https://mlsanity.com/paper/2510.09901/) (2025)
- [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026)
- [Synthelite: Chemist-aligned and feasibility-aware synthesis planning with LLMs](https://mlsanity.com/paper/2512.16424/) (2025)
- [SciCoQA: Quality Assurance for Scientific Paper--Code Alignment](https://mlsanity.com/paper/2601.12910/) (2026)
- [AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discovery](https://mlsanity.com/paper/2605.23204/) (2026)
- [AI Research Preference Models](https://mlsanity.com/paper/2608.13940/) (2026)
- [Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluator](https://mlsanity.com/paper/2507.11810/) (2025)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [LLM-Hanabi: Evaluating Multi-Agent Gameplays with Theory-of-Mind and Rationale Inference in Imperfect Information Collaboration Game](https://mlsanity.com/paper/2510.04980/) (2025)
- [AutoGraph-R1: End-to-End Reinforcement Learning for Knowledge Graph Construction](https://mlsanity.com/paper/2510.15339/) (2025)
- [Deep Ideation: Designing LLM Agents to Generate Novel Research Ideas on Scientific Concept Network](https://mlsanity.com/paper/2511.02238/) (2025)
- [Principle-Evolvable Scientific Discovery via Uncertainty Minimization](https://mlsanity.com/paper/2602.06448/) (2026)
- [ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning](https://mlsanity.com/paper/2511.14366/) (2025)
- [Effective Harness Engineering for Algorithm Discovery with Coding Agents](https://mlsanity.com/paper/2605.15221/) (2026)
- [Camyla: Scaling Autonomous Research in Medical Image Segmentation](https://mlsanity.com/paper/2604.10696/) (2026)
- [SciIntegrity-Bench: A Benchmark for Evaluating Academic Integrity in AI Scientist Systems](https://mlsanity.com/paper/2605.10246/) (2026)
- [Rethinking Prospect Theory for LLMs: Revealing the Instability of Decision-Making under Epistemic Uncertainty](https://mlsanity.com/paper/2508.08992/) (2025)
- [DixitWorld: Evaluating Multimodal Abductive Reasoning in Vision-Language Models with Multi-Agent Dixit Gameplay](https://mlsanity.com/paper/2510.10117/) (2025)
- [CritiCal: Can Critique Help LLM Uncertainty or Confidence Calibration?](https://mlsanity.com/paper/2510.24505/) (2025)
- [DataGovBench: Benchmarking LLM Agents for Real-World Data Governance Workflows](https://mlsanity.com/paper/2512.04416/) (2025)
- [VASP Agent: An Agentic Framework for Autonomous First-principles Calculations](https://mlsanity.com/paper/2512.19458/) (2025)
- [Sci-Reasoning: A Dataset Decoding AI Innovation Patterns](https://mlsanity.com/paper/2601.04577/) (2026)
- [HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning](https://mlsanity.com/paper/2601.22448/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are catalyzing a paradigm shift in scientific discovery, evolving from task-specific automation tools into increasingly autonomous agents and fundamentally redefining research processes and human-AI collaboration. This survey systematically charts this burgeoning field, placing a central focus on the changing roles and escalating capabilities of LLMs in science. Through the lens of the scientific method, we introduce a foundational three-level taxonomy-Tool, Analyst, and Scientist-to delineate their escalating autonomy and evolving responsibilities within the research lifecycle. We further identify pivotal challenges and future research trajectories such as robotic automation, self-improvement, and ethical governance. Overall, this survey provides a conceptual architecture and strategic foresight to navigate and shape the future of AI-driven scientific discovery, fostering both rapid innovation and responsible advancement. Github Repository: https://github.com/HKUST-KnowComp/Awesome-LLM-Scientific-Discovery.