mlsanityData to 14 Oct 2026

The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4

arXiv 2311.07361 ↗First posted 13 Nov 2023cs.CL cs.AIcited by 112

Not on the map. Why?

Abstract (arXiv)

In recent years, groundbreaking advancements in natural language processing have culminated in the emergence of powerful large language models (LLMs), which have showcased remarkable capabilities across a vast array of domains, including the understanding, generation, and translation of natural language, and even tasks that extend beyond language processing. In this report, we delve into the performance of LLMs within the context of scientific discovery, focusing on GPT-4, the state-of-the-art language model. Our investigation spans a diverse range of scientific areas encompassing drug discovery, biology, computational chemistry (density functional theory (DFT) and molecular dynamics (MD)), materials design, and partial differential equations (PDE). Evaluating GPT-4 on scientific tasks is crucial for uncovering its potential across various research domains, validating its domain-specific expertise, accelerating scientific progress, optimizing resource allocation, guiding future model development, and fostering interdisciplinary research. Our exploration methodology primarily consists of expert-driven case assessments, which offer qualitative insights into the model's comprehension of intricate scientific concepts and relationships, and occasionally benchmark testing, which quantitatively evaluates the model's capacity to solve well-defined domain-specific problems. Our preliminary exploration indicates that GPT-4 exhibits promising potential for a variety of scientific applications, demonstrating its aptitude for handling complex problem-solving and knowledge integration tasks. Broadly speaking, we evaluate GPT-4's knowledge base, scientific understanding, scientific numerical calculation abilities, and various scientific prediction capabilities.

Key facts. Cited by 112 papers in the map (data through 14 Oct 2026). Builds on 27 of its 115 references that are in the map. First posted on arXiv 13 Nov 2023.

Builds on

27 of 115 references.

References · 27

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  5. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  6. 2023PaLM 2 Technical Reportcited by 1,162
  7. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  8. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 668
  9. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  10. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
  11. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  12. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
15 more
  1. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 320
  2. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 265
  3. 2023MultiModal-GPT: A Vision and Language Model for Dialogue with Humanscited by 255
  4. 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 194
  5. 2023DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genomecited by 143
  6. 2023MathChat: Converse to Tackle Challenging Math Problems with LLM Agentscited by 68
  7. 2023AI-assisted coding: Experiments with GPT-4cited by 43
  8. 2023Are Large Language Models Ready for Healthcare? A Comparative Study on Clinical Language Understandingcited by 40
  9. 2021Dual-view Molecule Pre-trainingcited by 27
  10. 2023Towards Predicting Equilibrium Distributions for Molecular Systems with Deep Learningcited by 26
  11. 2023ChatMOF: An Autonomous AI System for Predicting and Generating Metal-Organic Frameworkscited by 17
  12. 2023Performance of ChatGPT on the US Fundamentals of Engineering Exam: Comprehensive Assessment of Proficiency and Potential Implications for Professional Environmental Engineering Practicecited by 15
  13. 2022MultiAuto-DeepONet: A Multi-resolution Autoencoder DeepONet for Nonlinear Dimension Reduction, Uncertainty Quantification and Operator Learning of Forward and Inverse Stochastic Problemscited by 13
  14. 2022GeneFormer: Learned Gene Compression using Transformer-based Context Modelingcited by 10
  15. 2022SSM-DTA: Breaking the Barriers of Data Scarcity in Drug-Target Affinity Predictioncited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 56; 2025: 35; 2026: 18.

112 papers.

Sort:

By area:

  1. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
  2. 2024ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Modelscited by 210
  3. 2024SciCode: A Research Coding Benchmark Curated by Scientistscited by 178
  4. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 148
  5. 2024LLM-SR: Scientific Equation Discovery via Programming with Large Language Modelscited by 133
  6. 2024PQCache: Product Quantization-based KVCache for Long Context LLM Inferencecited by 95
  7. 2024LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discoverycited by 86
  8. 2024Scientific Large Language Models: A Survey on Biological & Chemical Domainscited by 80
  9. 2024MLR-Copilot: Autonomous Machine Learning Research based on Large Language Models Agentscited by 67
  10. 2024Efficient Evolutionary Search Over Chemical Space with Large Language Modelscited by 61
  11. 2024HoneyComb: A Flexible LLM-Based Agent System for Materials Sciencecited by 57
  12. 2024BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuningcited by 54
38 more
  1. 2025ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflowscited by 52
  2. 2024A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?cited by 48
  3. 2024SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretationcited by 48
  4. 2024LLMatDesign: Autonomous Materials Discovery with Large Language Modelscited by 42
  5. 2024SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysiscited by 41
  6. 2024SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Modelscited by 39
  7. 2024ResearchTown: Simulator of Human Research Communitycited by 37
  8. 2024Are large language models superhuman chemists?cited by 34
  9. 2024Probing the limitations of multimodal language models for chemistry and materials researchcited by 32
  10. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 30
  11. 2024SciLitLLM: How to Adapt LLMs for Scientific Literature Understandingcited by 29
  12. 2024Evaluating LLMs' Divergent Thinking Capabilities for Scientific Idea Generation with Minimal Contextcited by 28
  13. 2025PiFlow: Principle-Aware Scientific Discovery with Multi-Agent Collaborationcited by 28
  14. 2024Is ChatGPT Transforming Academics' Writing Style?cited by 26
  15. 2024Materials science in the era of large language models: a perspectivecited by 24
  16. 2024Large Language Models in Drug Discovery and Development: From Disease Mechanisms to Clinical Trialscited by 24
  17. 2024Generating Interesting Scientific Ideas using Knowledge Graphs and LLMs: Evaluations with 100 Research Group Leaderscited by 22
  18. 2025BoxingGym: Benchmarking Progress in Automated Experimental Design and Model Discoverycited by 18
  19. 2023Controlling risks of AI in chemical science with agentscited by 17
  20. 2024Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Surveycited by 17
  21. 2024Quantum Many-Body Physics Calculations with Large Language Modelscited by 17
  22. 2025Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Modelcited by 17
  23. 2025Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoningcited by 17
  24. 2024Knowledge-guided Machine Learning: Current Trends and Future Prospectscited by 14
  25. 2025Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Modelscited by 14
  26. 2024Near to Mid-term Risks and Opportunities of Open-Source Generative AIcited by 13
  27. 2024Risks and Opportunities of Open-Source Generative AIcited by 13
  28. 2024Fast and Slow Generating: An Empirical Study on Large and Small Language Models Collaborative Decodingcited by 13
  29. 2026LLM-AutoSciLab: Closed-Loop Scientific Discovery via Active Experimentation with LLMscited by 12
  30. 2025Can Large Language Models Help Experimental Design for Causal Discovery?cited by 11
  31. 2024CAT-LLM: Style-enhanced Large Language Models with Text Style Definition for Chinese Article-style Transfercited by 10
  32. 2024Prioritizing High-Consequence Biological Capabilities in Evaluations of Artificial Intelligence Modelscited by 10
  33. 2024SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Typescited by 10
  34. 2025BioMaze: Benchmarking and Enhancing Large Language Models for Biological Pathway Reasoningcited by 10
  35. 20243D-MolT5: Leveraging Discrete Structural Information for Molecule-Text Modelingcited by 9
  36. 2024FABind+: Enhancing Molecular Docking through Improved Pocket Prediction and Pose Generationcited by 8
  37. 2024Towards Building Specialized Generalist AI with System 1 and System 2 Fusioncited by 8
  38. 2025ExLLM: Experience-Enhanced LLM Optimization for Molecular Design and Beyondcited by 8

50 most cited of 112 All 112 (JSON)