mlsanityData to 14 Oct 2026

How Far Are AI Scientists from Changing the World?

arXiv 2507.23276 ↗First posted 31 Jul 2025cs.AIcited by 15

Not on the map. Why?

Abstract (arXiv)

The emergence of large language models (LLMs) is propelling automated scientific discovery to the next level, with LLM-based Artificial Intelligence (AI) Scientist systems now taking the lead in scientific research. Several influential works have already appeared in the field of AI Scientist systems, with AI-generated research papers having been accepted at the ICLR 2025 workshop, suggesting that a human-level AI Scientist capable of uncovering phenomena previously unknown to humans, may soon become a reality. In this survey, we focus on the central question: How far are AI scientists from changing the world and reshaping the scientific research paradigm? To answer this question, we provide a prospect-driven review that comprehensively analyzes the current achievements of AI Scientist systems, identifying key bottlenecks and the critical components required for the emergence of a scientific agent capable of producing ground-breaking discoveries that solve grand challenges. We hope this survey will contribute to a clearer understanding of limitations of current AI Scientist systems, showing where we are, what is missing, and what the ultimate goals for scientific AI should be.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 143 of its 200 references that are in the map. First posted on arXiv 31 Jul 2025.

Builds on

143 of 200 references.

References · 143

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  7. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  10. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  11. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  12. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
38 more
  1. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  3. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  4. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  5. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  6. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  7. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
  8. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 769
  9. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  10. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  11. 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 493
  12. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 469
  13. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 425
  14. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 422
  15. 2025Accelerating scientific discovery with Co-Scientistcited by 401
  16. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  17. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
  18. 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 321
  19. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 319
  20. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 307
  21. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 302
  22. 2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challengescited by 282
  23. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 241
  24. 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 235
  25. 2025AIDE: AI-Driven Exploration in the Space of Codecited by 226
  26. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 206
  27. 2023SciMON: Scientific Inspiration Machines Optimized for Noveltycited by 179
  28. 2024SciCode: A Research Coding Benchmark Curated by Scientistscited by 178
  29. 2019BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarizationcited by 156
  30. 2023PaperQA: Retrieval-Augmented Generative Agent for Scientific Researchcited by 149
  31. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 148
  32. 2024Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Modelscited by 139
  33. 2024Language agents achieve superhuman synthesis of scientific knowledgecited by 137
  34. 2018A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applicationscited by 136
  35. 2020TLDR: Extreme Summarization of Scientific Documentscited by 136
  36. 2024DSBench: How Far Are Data Science Agents from Becoming Data Science Experts?cited by 125
  37. 2024CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmarkcited by 123
  38. 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 122

50 most cited of 143 All 143 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 11.

15 papers.

Sort:

By area:

  1. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 77
  2. 2025DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressivelycited by 63
  3. 2026AutoFigure: Generating and Refining Publication-Ready Scientific Illustrationscited by 23
  4. 2026AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discoverycited by 10
  5. 2026InnoEval: On Research Idea Evaluation as a Knowledge-Grounded, Multi-Perspective Reasoning Problemcited by 8
  6. 2025Towards autonomous quantum physics research using LLM agents with access to intelligent toolscited by 7
  7. 2026Principle-Evolvable Scientific Discovery via Uncertainty Minimizationcited by 7
  8. 2026Improving Coherence and Persistence in Agentic AI for System Optimizationcited by 7
  9. 2026Human Cognition in Machines: A Unified Perspective of World Modelscited by 5
  10. 2026What Do Evolutionary Coding Agents Evolve?cited by 3
  11. 2026Overview of the NLPCC 2026 Shared Task 11: Agent-Based Experiment Reproduction from Scientific Paperscited by 2
  12. 2026Playing ZendoWorld: Challenging AI Agents on Active Visual Concept Inductioncited by 1
3 more
  1. 2025CLINB: A Climate Intelligence Benchmark for Foundational Modelscited by 0
  2. 2026Position: AI Agents in Scientific Teams Should Be Studied as Human-Agent Systemscited by 0
  3. 2026How Is Automated Research Evaluated? A Survey of Benchmarks and Evaluation Practicescited by 0