mlsanityData to 14 Oct 2026

Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation

arXiv 2502.05151 ↗First posted 7 Feb 2025cs.CL cs.AI cs.CVcited by 42

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

With the advent of large multimodal language models, science is now at a threshold of an AI-based technological transformation. An emerging ecosystem of models and tools aims to support researchers throughout the scientific lifecycle, including (1) searching for relevant literature, (2) generating research ideas and conducting experiments, (3) producing text-based content, (4) creating multimodal artifacts such as figures and diagrams, and (5) evaluating scientific work, as in peer review. In this survey, we provide a curated overview of literature representative of the core techniques, evaluation practices, and emerging trends in AI-assisted scientific discovery. Across the five tasks outlined above, we discuss datasets, methods, results, evaluation strategies, limitations, and ethical concerns, including risks to research integrity through the misuse of generative models. We aim for this survey to serve both as an accessible, structured orientation for newcomers to the field, as well as a catalyst for new AI-based initiatives and their integration into future ``AI4Science'' systems.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 158 of its 335 references that are in the map. First posted on arXiv 7 Feb 2025.

Builds on

158 of 335 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Automatic benchmarking of large multimodal models via iterative experiment programming18 Jun

Other references · 157

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  5. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  6. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  8. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  9. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  10. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  11. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  12. 2016A Diagram Is Worth A Dozen Imagescited by 804
38 more
  1. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  2. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  3. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
  4. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 425
  5. 2019AutoML: A Survey of the State-of-the-Artcited by 390
  6. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 366
  7. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
  8. 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
  9. 2017A simple neural network module for relational reasoningcited by 346
  10. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 319
  11. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
  12. 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 268
  13. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 247
  14. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 238
  15. 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 235
  16. 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environmentscited by 231
  17. 2024ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Modelscited by 210
  18. 2023Can large language models provide useful feedback on research papers? A large-scale empirical analysiscited by 200
  19. 2023SciMON: Scientific Inspiration Machines Optimized for Noveltycited by 179
  20. 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 156
  21. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 148
  22. 2024LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMscited by 144
  23. 2024Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Modelscited by 139
  24. 2018A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applicationscited by 136
  25. 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generationcited by 129
  26. 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 122
  27. 2023Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systemscited by 121
  28. 2024Mapping the Increasing Use of LLMs in Scientific Paperscited by 118
  29. 2024MARG: Multi-Agent Review Generation for Scientific Paperscited by 104
  30. 2023Large Language Models for Automated Open-domain Scientific Hypotheses Discoverycited by 100
  31. 2024Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agentscited by 97
  32. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  33. 2022SciFact-Open: Towards open-domain scientific claim verificationcited by 94
  34. 2024CheckEval: A reliable LLM-as-a-Judge framework for evaluating text generation using checklistscited by 94
  35. 2024AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoMLcited by 87
  36. 2025LLM4SR: A Survey on Large Language Models for Scientific Researchcited by 83
  37. 2023AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZcited by 79
  38. 2020Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databasescited by 75

50 most cited of 157 All 157 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 24; 2026: 18.

42 papers.

Sort:

By area:

  1. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 422
  2. 2025Agentic Large Language Models, a surveycited by 108
  3. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 37
  4. 2025Step-by-Step Fact Verification System for Medical Claims with Explainable Reasoningcited by 17
  5. 2025CellForge: Agentic Design of Virtual Cell Modelscited by 13
  6. 2026PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writingcited by 13
  7. 2025LiTransProQA: an LLM-based Literary Translation evaluation metric with Professional Question Answeringcited by 10
  8. 2025LLLMs: A Data-Driven Survey of Evolving Research on Limitations of Large Language Modelscited by 9
  9. 2025Exploring Design of Multi-Agent LLM Dialogues for Research Ideationcited by 8
  10. 2026Real Money, Fake Models: Deceptive Model Claims in Shadow APIscited by 8
  11. 2026TikZilla: Scaling Text-to-TikZ with High-Quality Data and Reinforcement Learningcited by 7
  12. 2025Large Language Models Penetration in Scholarly Writing and Peer Reviewcited by 6
30 more
  1. 2025Knowledge Augmented Complex Problem Solving with Large Language Models: A Surveycited by 6
  2. 2025From Reasoning to Learning: A Survey on Hypothesis Discovery and Rule Learning with Large Language Modelscited by 5
  3. 2025Zhyper: Factorized Hypernetworks for Conditioned LLM Fine-Tuningcited by 5
  4. 2025PRISMM-Bench: A Benchmark of Peer-Review Grounded Multimodal Inconsistenciescited by 4
  5. 2026ScienceFlow: A long-horizon agent for ML research, scientific discovery and beyondcited by 4
  6. 2025Agents of Discoverycited by 3
  7. 2025CGBench: Benchmarking Language Model Scientific Reasoning for Clinical Genetics Researchcited by 3
  8. 2025On the Detectability of LLM-Generated Text: What Exactly Is LLM-Generated Text?cited by 3
  9. 2025QuantumBench: A Benchmark for Quantum Problem Solvingcited by 3
  10. 2025AutoPR: Let's Automate Your Academic Promotion!cited by 2
  11. 2025ResearchGPT: Benchmarking and Training LLMs for End-to-End Computer Science Research Workflowscited by 2
  12. 2026What Understanding Means in AI-Laden Astronomycited by 2
  13. 2025Enhancing Study-Level Inference from Clinical Trial Papers via Reinforcement Learning-Based Numeric Reasoningcited by 1
  14. 2025Generative AI and the future of scientometrics: current topics and future questionscited by 1
  15. 2026A unified multimodal understanding and generation model for cross-disciplinary scientific researchcited by 1
  16. 2026FMMD: A multimodal multidisciplinary dataset of open peer reviews from F1000Researchcited by 1
  17. 2026Beyond Via: Analysis and Estimation of the Impact of Large Language Models in Academic Paperscited by 1
  18. 2026Scientific discovery as meta-optimization: a combinatorial optimization case studycited by 1
  19. 2026The Greatness of Science Cannot Be Planned: Agentic Auto-Research is Fuzz Testingcited by 1
  20. 2026EdiTikZ: Scientific Figure Editing from Revision Trajectoriescited by 1
  21. 2026The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishingcited by 1
  22. 2025The Art of Audience Engagement: LLM-Based Thin-Slicing of Scientific Talkscited by 0
  23. 2025Charting the Future of Scholarly Knowledge with AI: A Community Perspectivecited by 0
  24. 2025CLINB: A Climate Intelligence Benchmark for Foundational Modelscited by 0
  25. 2026Story2Proposal: A Scaffold for Structured Scientific Paper Writingcited by 0
  26. 2026SciSchema.org: A Multidisciplinary Collection of Schemas for Structured Scientific Process Descriptionscited by 0
  27. 2026A Pathway to General-Purpose Scientific AI: Multimodal Comprehension of Scientific Imagescited by 0
  28. 2026Position: Want Better ML Reviews? Stop Asking Nicely and Start Incentivizing with a Credit Systemcited by 0
  29. 2026Deep Academic Survey: Stateful Agentic Closed-Loop Paradigm for Academic Survey Automationcited by 0
  30. 2026When Does Bigger Help? A Controlled Study of LLM Scale for Ontology Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Scientific Discovery with Generative AI: Progress, Opportunities, and Challenges
  2. 2026Contemporary AI lacks the imagination to diverge or negate in science
  3. 2025AI4Research: A Survey of Artificial Intelligence for Scientific Research
  4. 2025The Empowerment of Science of Science by Large Language Models: New Tools and Methods
  5. 2025AI-Researcher: Autonomous Scientific Innovation
  6. 2026Science Edge Evaluation: SEE the Missing Step Toward Real Scientific Discovery
  7. 2025Scientific Hypothesis Generation and Validation: Methods, Datasets, and Future Directions
  8. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery