Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluator
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are increasingly used in scientific research and discovery, supporting tasks ranging from literature retrieval and synthesis to hypothesis generation, autonomous experimentation, and research evaluation. Existing surveys often conflate scientific research with scientific discovery and typically organize systems by domain, task, or autonomy level alone. In this survey, we propose a four-role framework for understanding LLMs in scientific innovation: Assistant, Collaborator, Scientist, and Evaluator. The framework integrates three complementary dimensions: autonomy level, cognitive function, and scientific innovation, to distinguish research-oriented support from frontier-oriented discovery. We review representative methods, benchmarks, and evaluation practices for each role, examining their capabilities, limitations, and human oversight requirements. Across the literature, Assistant systems are comparatively mature in retrieval and synthesis but remain unreliable in open-ended applications; Collaborator systems expand the space of candidate hypotheses yet struggle with novelty-grounding trade-offs; Scientist systems increasingly automate research workflows but face reliability and safety bottlenecks; and Evaluator systems support review and verification while remaining weak in novelty assessment. We argue that progress in AI for science depends not only on model capability, but also on evaluation, oversight, accountability, and institutional integration.
Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 120 of its 207 references that are in the map. First posted on arXiv 16 Jul 2025.
Builds on
120 of 207 references.
References · 120
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
- 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
- 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
- 2020Fact or Fiction: Verifying Scientific Claimscited by 554
- 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 469
- 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 431
- 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 425
- 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 422
- 2022Multimodal Learning with Transformers: A Surveycited by 400
- 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 366
- 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
- 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 235
38 more
- 2020Explainable Automated Fact-Checking for Public Health Claimscited by 211
- 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 188
- 2020Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthycited by 183
- 2023SciMON: Scientific Inspiration Machines Optimized for Noveltycited by 179
- 2024SciCode: A Research Coding Benchmark Curated by Scientistscited by 178
- 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 175
- 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 164
- 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 156
- 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 148
- 2023TaskBench: Benchmarking Large Language Models for Task Automationcited by 144
- 2024Language agents achieve superhuman synthesis of scientific knowledgecited by 137
- 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derenderingcited by 133
- 2024DSBench: How Far Are Data Science Agents from Becoming Data Science Experts?cited by 125
- 2025MLGym: A New Framework and Benchmark for Advancing AI Research Agentscited by 124
- 2023ChartBench: A Benchmark for Complex Visual Reasoning in Chartscited by 120
- 2024AutoSurvey: Large Language Models Can Automatically Write Surveyscited by 119
- 2024Empowering Biomedical Discovery with AI Agentscited by 118
- 2024DiscoveryBench: Towards Data-Driven Discovery with Large Language Modelscited by 109
- 2023Large Language Models for Automated Open-domain Scientific Hypotheses Discoverycited by 100
- 2024DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agentscited by 100
- 2025DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Processcited by 100
- 2025AI-Researcher: Autonomous Scientific Innovationcited by 100
- 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
- 2022SciFact-Open: Towards open-domain scientific claim verificationcited by 94
- 2024AgentReview: Exploring Peer Review Dynamics with LLM Agentscited by 93
- 2025From Automation to Autonomy: A Survey on Large Language Models in Scientific Discoverycited by 93
- 2023Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinementcited by 89
- 2024LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discoverycited by 86
- 2025LLM4SR: A Survey on Large Language Models for Scientific Researchcited by 83
- 2024ChartGemma: Visual Instruction-tuning for Chart Reasoning in the Wildcited by 79
- 2024MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypothesescited by 72
- 2025AgentRxiv: Towards Collaborative Autonomous Researchcited by 72
- 2024Large language models surpass human experts in predicting neuroscience resultscited by 69
- 2024The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Ratescited by 69
- 2024ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoningcited by 68
- 2024Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent Systemcited by 68
- 2024MLR-Copilot: Autonomous Machine Learning Research based on Large Language Models Agentscited by 67
- 2025PaSa: An LLM Agent for Comprehensive Academic Paper Searchcited by 64
50 most cited of 120 All 120 (JSON)
Built on by
Citing papers per year: 2025: 4; 2026: 5.
9 papers.
- 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 77
- 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 37
- 2026OpenNovelty: An LLM-powered Agentic System for Verifiable Scholarly Novelty Assessmentcited by 8
- 2025ReviewGuard: Enhancing Deficient Peer Review Detection via LLM-Driven Data Augmentationcited by 3
- 2026T2S-Bench & Structure-of-Thought: Benchmarking and Prompting Comprehensive Text-to-Structure Reasoningcited by 2
- 2025Large Language Models Meet Virtual Cell: A Surveycited by 1
- 2026Mimosa Framework: Toward Evolving Multi-Agent Systems for Scientific Researchcited by 1
- 2026DataScribe: An AI-Native, Policy-Aligned Web Platform for Multi-Objective Materials Design and Discoverycited by 0
- 2026OUTLINEFORGE: Hierarchical Reinforcement Learning with Explicit States for Scientific Writingcited by 0
- 2026Mimosa Framework: Toward Evolving Multi-Agent Systems for Scientific Research30 Mar 2026
- 2026T2S-Bench & Structure-of-Thought: Benchmarking and Prompting Comprehensive Text-to-Structure Reasoning4 Mar 2026
- 2026OUTLINEFORGE: Hierarchical Reinforcement Learning with Explicit States for Scientific Writing14 Jan 2026
- 2026DataScribe: An AI-Native, Policy-Aligned Web Platform for Multi-Objective Materials Design and Discovery12 Jan 2026
- 2026OpenNovelty: An LLM-powered Agentic System for Verifiable Scholarly Novelty Assessment4 Jan 2026
- 2025ReviewGuard: Enhancing Deficient Peer Review Detection via LLM-Driven Data Augmentation18 Oct 2025
- 2025Large Language Models Meet Virtual Cell: A Survey9 Oct 2025
- 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers28 Aug 2025
- 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery18 Aug 2025