mlsanityData to 7 Oct 2026

Large Language Models for Bioinformatics

arXiv 2501.06271 ↗First posted 10 Jan 2025q-bio.QM cs.AI cs.CEcited by 3

Not on the map. Why?

Abstract (arXiv)

With the rapid advancements in large language model (LLM) technology and the emergence of bioinformatics-specific language models (BioLMs), there is a growing need for a comprehensive analysis of the current landscape, computational characteristics, and diverse applications. This survey aims to address this need by providing a thorough review of BioLMs, focusing on their evolution, classification, and distinguishing features, alongside a detailed examination of training methodologies, datasets, and evaluation frameworks. We explore the wide-ranging applications of BioLMs in critical areas such as disease diagnosis, drug discovery, and vaccine development, highlighting their impact and transformative potential in bioinformatics. We identify key challenges and limitations inherent in BioLMs, including data privacy and security concerns, interpretability issues, biases in training data and model outputs, and domain adaptation complexities. Finally, we highlight emerging trends and future directions, offering valuable insights to guide researchers and clinicians toward advancing BioLMs for increasingly sophisticated biological and clinical applications.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 129 of its 391 references that are in the map. First posted on arXiv 10 Jan 2025.

Builds on

129 of 391 references.

References · 129

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2024The Llama 3 Herd of Modelscited by 14,192
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  9. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  10. 2023Visual Instruction Tuningcited by 7,792
  11. 2017Deep reinforcement learning from human preferencescited by 3,846
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
38 more
  1. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  2. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  3. 2019Language Models as Knowledge Bases?cited by 1,922
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  5. 2016Concrete Problems in AI Safetycited by 1,582
  6. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  7. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  8. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  9. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  10. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  11. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  12. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
  13. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  14. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  15. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  16. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 388
  17. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  18. 2017A Structured Self-attentive Sentence Embeddingcited by 318
  19. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  20. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  21. 2023Challenges and Applications of Large Language Modelscited by 251
  22. 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 244
  23. 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 212
  24. 2023HuatuoGPT, towards Taming Language Model to Be a Doctorcited by 211
  25. 2023BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Taskscited by 201
  26. 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 187
  27. 2020ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computingcited by 162
  28. 2020A Review of Generalized Zero-Shot Learning Methodscited by 159
  29. 2021On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptationcited by 147
  30. 2024Continual Learning for Large Language Models: A Surveycited by 144
  31. 2023DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genomecited by 139
  32. 2020ProGen: Language Modeling for Protein Generationcited by 127
  33. 2024Evaluation of OpenAI o1: Opportunities and Challenges of AGIcited by 127
  34. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 124
  35. 2023Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Healthcited by 118
  36. 2024On Protecting the Data Privacy of Large Language Models (LLMs): A Surveycited by 91
  37. 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectivescited by 88
  38. 2024Scientific Large Language Models: A Survey on Biological & Chemical Domainscited by 80

50 most cited of 129 All 129 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.