mlsanityData to 14 Oct 2026

Protein Large Language Models: A Comprehensive Survey

arXiv 2502.17504 ↗First posted 21 Feb 2025q-bio.BM cs.AI cs.CEcited by 18

Not on the map. Why?

Abstract (arXiv)

Protein-specific large language models (Protein LLMs) are revolutionizing protein science by enabling more efficient protein structure prediction, function annotation, and design. While existing surveys focus on specific aspects or applications, this work provides the first comprehensive overview of Protein LLMs, covering their architectures, training datasets, evaluation metrics, and diverse applications. Through a systematic analysis of over 100 articles, we propose a structured taxonomy of state-of-the-art Protein LLMs, analyze how they leverage large-scale protein sequence data for improved accuracy, and explore their potential in advancing protein engineering and biomedical research. Additionally, we discuss key challenges and future directions, positioning Protein LLMs as essential tools for scientific discovery in protein science. Resources are maintained at https://github.com/Yijia-Xiao/Protein-LLM-Survey.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 57 of its 168 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

57 of 168 references.

References · 57

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  8. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  9. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  10. 2019Axial Attention in Multidimensional Transformerscited by 358
  11. 2020Learning from Protein Structure with Geometric Vector Perceptronscited by 313
  12. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 285
38 more
  1. 2019Evaluating Protein Transfer Learning with TAPEcited by 231
  2. 2022Protein Representation Learning by Geometric Structure Pretrainingcited by 162
  3. 2023BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associationscited by 100
  4. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  5. 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
  6. 2020BERTology Meets Biology: Interpreting Attention in Protein Language Modelscited by 85
  7. 2022BioBART: Pretraining and Evaluation of A Biomedical Generative Language Modelcited by 83
  8. 2024Scientific Large Language Models: A Survey on Biological & Chemical Domainscited by 80
  9. 2022Tranception: protein fitness prediction with autoregressive transformers and inference-time retrievalcited by 79
  10. 2023ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Textscited by 78
  11. 2022PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understandingcited by 77
  12. 2022OntoProtein: Protein Pretraining With Gene Ontology Embeddingcited by 70
  13. 2023Structure-informed Language Models Are Protein Designerscited by 68
  14. 2024InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoderscited by 66
  15. 2023A Text-guided Protein Design Frameworkcited by 64
  16. 2024BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuningcited by 54
  17. 2024ProLLaMA: A Protein Large Language Model for Multi-Task Protein Language Processingcited by 53
  18. 2019Learning protein sequence embeddings using information from structurecited by 51
  19. 2024xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Proteincited by 50
  20. 2023Ankh: Optimized Protein Language Model Unlocks General-Purpose Modellingcited by 45
  21. 2022RITA: a Study on Scaling Up Generative Protein Sequence Modelscited by 42
  22. 2023Prot2Text: Multimodal Protein's Function Generation with GNNs and Transformerscited by 32
  23. 2023InstructProtein: Aligning Human and Protein Language via Knowledge Instructioncited by 28
  24. 2024ProtT3: Protein-to-Text Generation for Text-based Protein Understandingcited by 25
  25. 2021Fold2Seq: A Joint Sequence(1D)-Fold(3D) Embedding-based Generative Model for Protein Designcited by 23
  26. 2021Modeling Protein Using Large-scale Pretrain Language Modelcited by 21
  27. 2023PeptideBERT: A Language Model based on Transformers for Peptide Property Predictioncited by 21
  28. 2024ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Trainingcited by 21
  29. 2022Exploring evolution-aware & -free protein language models as protein function predictorscited by 20
  30. 2024A Fine-tuning Dataset and Benchmark for Large Language Models for Protein Understandingcited by 19
  31. 2019Accelerated Nuclear Magnetic Resonance Spectroscopy with Deep Learningcited by 16
  32. 2023A Systematic Study of Joint Representation Learning on Protein Sequences and Structurescited by 15
  33. 2024From Words to Molecules: A Survey of Large Language Models in Chemistrycited by 14
  34. 2024Endowing Protein Language Models with Structural Knowledgecited by 11
  35. 2024Annotation-guided Protein Design with Multi-Level Domain Alignmentcited by 9
  36. 2024Geneverse: A collection of Open-source Multimodal Large Language Models for Genomic and Proteomic Researchcited by 9
  37. 2024TourSynbio: A Multi-Modal Large Model and Agent Framework to Bridge Text and Protein Sequences for Protein Engineeringcited by 9
  38. 2024RNA-GPT: Multimodal Generative System for RNA Sequence Understandingcited by 8

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 8.

18 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  2. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 37
  3. 2025How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlookcited by 20
  4. 2025MD-LLM-1: A Large Language Model for Molecular Dynamicscited by 7
  5. 2025Advances in Protein Representation Learning: Methods, Applications, and Future Directionscited by 3
  6. 2025Resilient Biosecurity in the Era of AI-Enabled Bioweaponscited by 3
  7. 2025Constrained Sliced Wasserstein Embeddingcited by 2
  8. 2026Protein Language Models Diverge from Natural Language: Comparative Analysis and Improved Inferencecited by 2
  9. 2026ProtRLSearch: A Multi-Round Multimodal Protein Search Agent with Large Language Models Trained via Reinforcement Learningcited by 1
  10. 2026When Context Bites: Detecting RAG Poisoning via Document-Level Attention Collapsecited by 1
  11. 2025Protein as a Second Language for LLMscited by 0
  12. 2025g-DPO: Scalable Preference Optimization for Protein Language Modelscited by 0
6 more
  1. 2025Reflection Pretraining Enables Token-Level Self-Correction in Biological Sequence Modelscited by 0
  2. 2026Know Your Scientist: KYC as Biosecurity Infrastructurecited by 0
  3. 2026ES-Merging: Biological MLLM Merging via Embedding Space Signalscited by 0
  4. 2026From Syntax to Semantics: Unveiling the Emergence of Chirality in SMILES Translation Modelscited by 0
  5. 2026Enhancing Protein Representation Learning via Manifold Restore Mixingcited by 0
  6. 2026The Illusion of Replacement: Rethinking Specialized Machine Learning Models in the Foundation Model Eracited by 0