mlsanityData to 7 Oct 2026

Scientific Large Language Models: A Survey on Biological & Chemical Domains

arXiv 2401.14656 ↗First posted 26 Jan 2024cs.CLcited by 80

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have emerged as a transformative power in enhancing natural language comprehension, representing a significant stride toward artificial general intelligence. The application of LLMs extends beyond conventional linguistic boundaries, encompassing specialized linguistic systems developed within various scientific disciplines. This growing interest has led to the advent of scientific LLMs, a novel subclass specifically engineered for facilitating scientific discovery. As a burgeoning area in the community of AI for Science, scientific LLMs warrant comprehensive exploration. However, a systematic and up-to-date survey introducing them is currently lacking. In this paper, we endeavor to methodically delineate the concept of "scientific language", whilst providing a thorough review of the latest advancements in scientific LLMs. Given the expansive realm of scientific disciplines, our analysis adopts a focused lens, concentrating on the biological and chemical domains. This includes an in-depth examination of LLMs for textual knowledge, small molecules, macromolecular proteins, genomic sequences, and their combinations, analyzing them in terms of model architectures, capabilities, datasets, and evaluation. Finally, we critically examine the prevailing challenges and point out promising research directions along with the advances of LLMs. By offering a comprehensive overview of technical developments in this field, this survey aspires to be an invaluable resource for researchers navigating the intricate landscape of scientific LLMs.

Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). Builds on 87 of its 264 references that are in the map. First posted on arXiv 26 Jan 2024.

Builds on

87 of 264 references.

References · 87

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  9. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  12. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
38 more
  1. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  2. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  3. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  4. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  5. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  6. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  7. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  8. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  9. 2022Galactica: A Large Language Model for Sciencecited by 736
  10. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  11. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  12. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
  13. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  14. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 332
  15. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  16. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 294
  17. 2018Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Modelscited by 270
  18. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  19. 2020GEOM: Energy-annotated molecular conformations for property prediction and molecular generationcited by 238
  20. 2019Evaluating Protein Transfer Learning with TAPEcited by 229
  21. 2018Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Predictioncited by 225
  22. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 209
  23. 2021Large-Scale Chemical Language Representations Capture Molecular Structure and Propertiescited by 203
  24. 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 187
  25. 2022Talking About Large Language Modelscited by 174
  26. 2022Translation between Molecules and Natural Languagecited by 173
  27. 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 165
  28. 2020ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computingcited by 162
  29. 2022ProGen2: Exploring the Boundaries of Protein Language Modelscited by 156
  30. 2023DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genomecited by 139
  31. 2023Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Modelscited by 131
  32. 2022A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Languagecited by 129
  33. 2020ProGen: Language Modeling for Protein Generationcited by 127
  34. 2023The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4cited by 112
  35. 2023BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associationscited by 100
  36. 2023BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicinecited by 99
  37. 2020Molecule Attention Transformercited by 92
  38. 2023Unifying Molecular and Textual Representations via Multi-task Language Modellingcited by 92

50 most cited of 87 All 87 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 24; 2025: 43; 2026: 13.

80 papers.

Sort:

By area:

  1. 2024Discrete Flow Matchingcited by 301
  2. 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 153
  3. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  4. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 96
  5. 2025El Agente: An Autonomous Agent for Quantum Chemistrycited by 43
  6. 2024FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answeringcited by 38
  7. 2025BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Modelcited by 35
  8. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  9. 2025PiFlow: Principle-Aware Scientific Discovery with Multi-Agent Collaborationcited by 25
  10. 2026Intern-S1-Pro: Scientific Multimodal Foundation Model at Trillion Scalecited by 25
  11. 2024SciQAG: A Framework for Auto-Generated Science Question Answering Dataset with Fine-grained Evaluationcited by 23
  12. 2025Harnessing Large Language Models for Disaster Management: A Surveycited by 20
38 more
  1. 2025Protein Large Language Models: A Comprehensive Surveycited by 18
  2. 2025RAG-Enhanced Collaborative LLM Agents for Drug Discoverycited by 18
  3. 2024Benchmarking Large Language Models for Molecule Prediction Taskscited by 17
  4. 2024SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Taskscited by 17
  5. 2025Flow Matching Meets Biology and Life Science: A Surveycited by 14
  6. 2024Bio-xLSTM: Generative modeling, representation and in-context learning of biological and chemical sequencescited by 13
  7. 2025PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priorscited by 13
  8. 2025Efficient Training-Free Online Routing for High-Volume Multi-LLM Servingcited by 13
  9. 2024From Generalist to Specialist: A Survey of Large Language Models for Chemistrycited by 11
  10. 2025Large Means Left: Political Bias in Large Language Models Increases with Their Number of Parameterscited by 11
  11. 2024Geometry Informed Tokenization of Molecules for Language Model Generationcited by 10
  12. 2024Advancing biomolecular understanding and design following human instructionscited by 10
  13. 20243D-MolT5: Leveraging Discrete Structural Information for Molecule-Text Modelingcited by 9
  14. 2024Scalable Training of Trustworthy and Energy-Efficient Predictive Graph Foundation Models for Atomistic Materials Modeling: A Case Study with HydraGNNcited by 8
  15. 2024Towards Building Specialized Generalist AI with System 1 and System 2 Fusioncited by 8
  16. 2025A Multimodal PDE Foundation Model for Prediction and Scientific Text Descriptionscited by 8
  17. 2025Benchmarking Large Language Models on Multiple Tasks in Bioinformatics NLP with Promptingcited by 8
  18. 2024Harnessing Large Language Models as Post-hoc Correctorscited by 7
  19. 2024CARE: a Benchmark Suite for the Classification and Retrieval of Enzymescited by 7
  20. 2024LLM4GRN: Discovering Causal Gene Regulatory Networks with LLMs -- Evaluation through Synthetic Data Generationcited by 7
  21. 2025PDE-Agent: A toolchain-augmented multi-agent framework for PDE solvingcited by 7
  22. 2026Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMscited by 7
  23. 2025Computational Protein Science in the Era of Large Language Models (LLMs)cited by 6
  24. 2025PriM: Principle-Inspired Material Discovery through Multi-Agent Collaborationcited by 6
  25. 2025How to Detect and Defeat Molecular Mirage: A Metric-Driven Benchmark for Hallucination in LLM-based Molecular Comprehensioncited by 6
  26. 2025Toward Scientific Reasoning in LLMs: Training from Expert Discussions via Reinforcement Learningcited by 6
  27. 2025BioMARS: A Multi-Agent Robotic System for Autonomous Biological Experimentscited by 6
  28. 2025LingGym: How Far Are LLMs from Thinking Like Field Linguists?cited by 6
  29. 2025TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Taskscited by 5
  30. 2025An LLM-enabled Multi-Agent Autonomous Mechatronics Design Frameworkcited by 4
  31. 2025BioMol-MQA: A Multi-Modal Question Answering Dataset For LLM Reasoning Over Bio-Molecular Interactionscited by 4
  32. 2025Towards the Autonomous Optimization of Urban Logistics: Training Generative AI with Scientific Tools via Agentic Digital Twins and Model Context Protocolcited by 4
  33. 2026Agentic reinforcement learning empowers next-generation chemical language models for molecular design and synthesiscited by 4
  34. 2024Quo Vadis ChatGPT? From Large Language Models to Large Knowledge Modelscited by 3
  35. 2025Large Language Models for Bioinformaticscited by 3
  36. 2025Are All Prompt Components Value-Neutral? Understanding the Heterogeneous Adversarial Robustness of Dissected Prompt in Large Language Modelscited by 3
  37. 2025MolEdit: Knowledge Editing for Multimodal Molecule Language Modelscited by 3
  38. 2024Large Vision-Language Models for Remote Sensing Visual Question Answeringcited by 2

50 most cited of 80 All 80 (JSON)