mlsanityData to 14 Oct 2026

Foundation Models for AI-Enabled Biological Design

arXiv 2505.11610 ↗First posted 16 May 2025cs.AI cs.LG q-bio.BMcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper surveys foundation models for AI-enabled biological design, focusing on recent developments in applying large-scale, self-supervised models to tasks such as protein engineering, small molecule design, and genomic sequence design. Though this domain is evolving rapidly, this survey presents and discusses a taxonomy of current models and methods. The focus is on challenges and solutions in adapting these models for biological applications, including biological sequence modeling architectures, controllability in generation, and multi-modal integration. The survey concludes with a discussion of open problems and future directions, offering concrete next-steps to improve the quality of biological sequence generation.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 29 of its 65 references that are in the map. First posted on arXiv 16 May 2025.

Builds on

29 of 65 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  9. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  10. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  11. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  12. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
17 more
  1. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 499
  2. 2021Explaining Neural Scaling Lawscited by 363
  3. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  4. 2019Evaluating Protein Transfer Learning with TAPEcited by 231
  5. 2021Autoregressive Diffusion Modelscited by 210
  6. 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 194
  7. 2022ProGen2: Exploring the Boundaries of Protein Language Modelscited by 159
  8. 2023Protein Design with Guided Discrete Diffusioncited by 157
  9. 2023MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adaptercited by 128
  10. 2022Regression Transformer: Concurrent sequence regression and generation for molecular language modelingcited by 50
  11. 2023nach0: Multimodal Natural and Chemical Languages Foundation Modelcited by 26
  12. 2022The Role of AI in Drug Discovery: Challenges, Opportunities, and Strategiescited by 20
  13. 2024Genomic Language Models: Opportunities and Challengescited by 20
  14. 2024DiscDiff: Latent Diffusion Model for DNA Sequence Generationcited by 12
  15. 2024Advancing biomolecular understanding and design following human instructionscited by 10
  16. 2023Towards Joint Sequence-Structure Generation of Nucleic Acid and Protein Complexes with SE(3)-Discrete Diffusioncited by 9
  17. 2024Protein-Mamba: Biological Mamba Models for Protein Function Predictioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Advances in Machine Learning for Directed Evolution: A Five-Year Retrospectivecited by 0