# Towards Scientific Discovery with Generative AI: Progress, Opportunities, and Challenges

arXiv: https://arxiv.org/abs/2412.11427 · HTML: https://mlsanity.com/paper/2412.11427/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 67 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 30 of 38)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [ChemCrow: Augmenting large-language models with chemistry tools](https://mlsanity.com/paper/2304.05376/) (2023)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [LeanDojo: Theorem Proving with Retrieval-Augmented Language Models](https://mlsanity.com/paper/2306.15626/) (2023)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [Generative Language Modeling for Automated Theorem Proving](https://mlsanity.com/paper/2009.03393/) (2020)
- [Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers](https://mlsanity.com/paper/2409.04109/) (2024)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs](https://mlsanity.com/paper/2210.12283/) (2022)
- [Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks](https://mlsanity.com/paper/2307.02477/) (2023)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021)
- [Learning to Plan Chemical Syntheses](https://mlsanity.com/paper/1708.04202/) (2017)
- [SciMON: Scientific Inspiration Machines Optimized for Novelty](https://mlsanity.com/paper/2305.14259/) (2023)
- [AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity](https://mlsanity.com/paper/2006.10782/) (2020)
- [Proof Artifact Co-training for Theorem Proving with Language Models](https://mlsanity.com/paper/2102.06203/) (2021)
- [Semantic Probabilistic Layers for Neuro-Symbolic Learning](https://mlsanity.com/paper/2206.00426/) (2022)
- [SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning](https://mlsanity.com/paper/2409.05556/) (2024)
- [GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text](https://mlsanity.com/paper/2308.06911/) (2023)
- [Transformer-based Planning for Symbolic Regression](https://mlsanity.com/paper/2303.06833/) (2023)

## Built on by (most cited first, 30 of 41)

- [AI-Researcher: Autonomous Scientific Innovation](https://mlsanity.com/paper/2505.18705/) (2025)
- [LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models](https://mlsanity.com/paper/2504.10415/) (2025)
- [EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery](https://mlsanity.com/paper/2603.08127/) (2026)
- [Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning](https://mlsanity.com/paper/2502.02871/) (2025)
- [PiFlow: Principle-Aware Scientific Discovery with Multi-Agent Collaboration](https://mlsanity.com/paper/2505.15047/) (2025)
- [PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priors](https://mlsanity.com/paper/2507.15550/) (2025)
- [Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systems](https://mlsanity.com/paper/2505.17968/) (2025)
- [LLEMA: Evolutionary Search with LLMs for Multi-Objective Materials Discovery](https://mlsanity.com/paper/2510.22503/) (2025)
- [Agentic Deep Graph Reasoning Yields Self-Organizing Knowledge Networks](https://mlsanity.com/paper/2502.13025/) (2025)
- [LLM-AutoSciLab: Closed-Loop Scientific Discovery via Active Experimentation with LLMs](https://mlsanity.com/paper/2605.24043/) (2026)
- [Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physics](https://mlsanity.com/paper/2510.09901/) (2025)
- [AtmosSci-Bench: Evaluating the Recent Advance of Large Language Model for Atmospheric Science](https://mlsanity.com/paper/2502.01159/) (2025)
- [Benchmarking AI scientists for omics data driven biological discovery](https://mlsanity.com/paper/2505.08341/) (2025)
- [Mimicking the Physicist's Eye:A VLM-centric Approach for Physics Formula Discovery](https://mlsanity.com/paper/2508.17380/) (2025)
- [The Need for Verification in AI-Driven Scientific Discovery](https://mlsanity.com/paper/2509.01398/) (2025)
- [Deep Ideation: Designing LLM Agents to Generate Novel Research Ideas on Scientific Concept Network](https://mlsanity.com/paper/2511.02238/) (2025)
- [Towards a Medical AI Scientist](https://mlsanity.com/paper/2603.28589/) (2026)
- [LLM-ACES: Closed-Loop Discovery of Dynamical Systems with LLM-Guided Adaptive Search](https://mlsanity.com/paper/2606.25039/) (2026)
- [Diffusion Models for Future Networks and Communications: A Comprehensive Survey](https://mlsanity.com/paper/2508.01586/) (2025)
- [ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning](https://mlsanity.com/paper/2511.14366/) (2025)
- [AI-Newton: A Concept-Driven Physical Law Discovery System without Prior Physical Knowledge](https://mlsanity.com/paper/2504.01538/) (2025)
- [Geometry-Calibrated Conformal Abstention for Language Models](https://mlsanity.com/paper/2604.27914/) (2026)
- [AI-Driven Automation Can Become the Foundation of Next-Era Science of Science Research](https://mlsanity.com/paper/2505.12039/) (2025)
- [ProjectionBench: Evaluating Scientific Hypothesis Generation in LLMs Under Progressive Information Disclosure](https://mlsanity.com/paper/2605.30284/) (2026)
- [Position: Academic Conferences are Potentially Facing Denominator Gaming Caused by Fully Automated Scientific Agents](https://mlsanity.com/paper/2605.09915/) (2026)
- [SURFACEBENCH: A Geometry-Aware Benchmark for Symbolic Surface Discovery](https://mlsanity.com/paper/2511.10833/) (2025)
- [From Verification Burden to Trusted Collaboration: Design Goals for LLM-Assisted Literature Reviews](https://mlsanity.com/paper/2512.11661/) (2025)
- [SciHorizon-GENE: Benchmarking LLM for Life Sciences Inference from Gene Knowledge to Functional Understanding](https://mlsanity.com/paper/2601.12805/) (2026)
- [Beyond the Desk: Barriers and Future Opportunities for AI to Assist Scientists in Embodied Physical Tasks](https://mlsanity.com/paper/2603.19504/) (2026)
- [AwesomeLit: Towards Hypothesis Generation with Agent-Supported Literature Research](https://mlsanity.com/paper/2603.22648/) (2026)

## Abstract (arXiv)

Scientific discovery is a complex cognitive process that has driven human knowledge and technological progress for centuries. While artificial intelligence (AI) has made significant advances in automating aspects of scientific reasoning, simulation, and experimentation, we still lack integrated AI systems capable of performing autonomous long-term scientific research and discovery. This paper examines the current state of AI for scientific discovery, highlighting recent progress in large language models and other AI techniques applied to scientific tasks. We then outline key challenges and promising research directions toward developing more comprehensive AI systems for scientific discovery, including the need for science-focused AI agents, improved benchmarks and evaluation metrics, multimodal scientific representations, and unified frameworks combining reasoning, theorem proving, and data-driven modeling. Addressing these challenges could lead to transformative AI tools to accelerate progress across disciplines towards scientific discovery.