Large Language Models for Bioinformatics
Not on the map. Why?
Abstract (arXiv)
With the rapid advancements in large language model (LLM) technology and the emergence of bioinformatics-specific language models (BioLMs), there is a growing need for a comprehensive analysis of the current landscape, computational characteristics, and diverse applications. This survey aims to address this need by providing a thorough review of BioLMs, focusing on their evolution, classification, and distinguishing features, alongside a detailed examination of training methodologies, datasets, and evaluation frameworks. We explore the wide-ranging applications of BioLMs in critical areas such as disease diagnosis, drug discovery, and vaccine development, highlighting their impact and transformative potential in bioinformatics. We identify key challenges and limitations inherent in BioLMs, including data privacy and security concerns, interpretability issues, biases in training data and model outputs, and domain adaptation complexities. Finally, we highlight emerging trends and future directions, offering valuable insights to guide researchers and clinicians toward advancing BioLMs for increasingly sophisticated biological and clinical applications.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 129 of its 391 references that are in the map. First posted on arXiv 10 Jan 2025.
Builds on
129 of 391 references.
References · 129
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2023Visual Instruction Tuningcited by 7,792
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
38 more
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
- 2019Language Models as Knowledge Bases?cited by 1,922
- 2022Large Language Models Encode Clinical Knowledgecited by 1,807
- 2016Concrete Problems in AI Safetycited by 1,582
- 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
- 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
- 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
- 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
- 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
- 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
- 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
- 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
- 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
- 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
- 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 388
- 2021Pre-Trained Models: Past, Present and Futurecited by 319
- 2017A Structured Self-attentive Sentence Embeddingcited by 318
- 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
- 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
- 2023Challenges and Applications of Large Language Modelscited by 251
- 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 244
- 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 212
- 2023HuatuoGPT, towards Taming Language Model to Be a Doctorcited by 211
- 2023BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Taskscited by 201
- 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 187
- 2020ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computingcited by 162
- 2020A Review of Generalized Zero-Shot Learning Methodscited by 159
- 2021On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptationcited by 147
- 2024Continual Learning for Large Language Models: A Surveycited by 144
- 2023DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genomecited by 139
- 2020ProGen: Language Modeling for Protein Generationcited by 127
- 2024Evaluation of OpenAI o1: Opportunities and Challenges of AGIcited by 127
- 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 124
- 2023Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Healthcited by 118
- 2024On Protecting the Data Privacy of Large Language Models (LLMs): A Surveycited by 91
- 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectivescited by 88
- 2024Scientific Large Language Models: A Survey on Biological & Chemical Domainscited by 80
50 most cited of 129 All 129 (JSON)
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
- 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 34
- 2026MechPert: Mechanistic Consensus as an Inductive Bias for Unseen Perturbation Predictioncited by 1
- 2025Generative Artificial Intelligence in Bioinformatics: A Systematic Review of Models, Applications, and Methodological Advancescited by 0
- 2026MechPert: Mechanistic Consensus as an Inductive Bias for Unseen Perturbation Prediction14 Feb 2026
- 2025Generative Artificial Intelligence in Bioinformatics: A Systematic Review of Models, Applications, and Methodological Advances5 Nov 2025
- 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers28 Aug 2025