# BioNeMo Framework: a modular, high-performance library for AI model development in drug discovery

arXiv: https://arxiv.org/abs/2411.10548 · HTML: https://mlsanity.com/paper/2411.10548/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 27 references that are in the map.
- First posted on arXiv 15 Nov 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM](https://mlsanity.com/paper/2104.04473/) (2021)
- [GEOM: Energy-annotated molecular conformations for property prediction and molecular generation](https://mlsanity.com/paper/2006.05531/) (2020)
- [NeMo: a toolkit for building AI applications using Neural Modules](https://mlsanity.com/paper/1909.09577/) (2019)
- [ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing](https://mlsanity.com/paper/2007.06225/) (2020)
- [Reinforcement learning on structure-conditioned categorical diffusion for protein inverse folding](https://mlsanity.com/paper/2410.17173/) (2024)

## Built on by (most cited first, 10 of 10)

- [Accelerating drug discovery with Artificial: a whole-lab orchestration and scheduling system for self-driving labs](https://mlsanity.com/paper/2504.00986/) (2025)
- [scDataset: Scalable Data Loading for Deep Learning on Large-Scale Single-Cell Omics](https://mlsanity.com/paper/2506.01883/) (2025)
- [Contrastive Mutual Information Learning: Toward Robust Representations without Positive-Pair Augmentations](https://mlsanity.com/paper/2509.21511/) (2025)
- [Steering an Active Learning Workflow Towards Novel Materials Discovery via Queue Prioritization](https://mlsanity.com/paper/2509.25538/) (2025)
- [Best Practices for Biorisk Evaluations on Open-Weight Bio-Foundation Models](https://mlsanity.com/paper/2510.27629/) (2025)
- [Scalable Agentic Reasoning for Designing Biologics Targeting Intrinsically Disordered Proteins](https://mlsanity.com/paper/2512.15930/) (2025)
- [SCALE:Scalable Conditional Atlas-Level Endpoint transport for virtual cell perturbation prediction](https://mlsanity.com/paper/2603.17380/) (2026)
- [annbatch unlocks terabyte-scale training of biological data in anndata](https://mlsanity.com/paper/2604.01949/) (2026)
- [MultiMolecule: a modular ecosystem for biomolecular sequence-model workflows](https://mlsanity.com/paper/2606.16540/) (2026)
- [Native Multi-Dimensional Subquadratic Operators via Input Dependent Long Convolutions](https://mlsanity.com/paper/2607.19378/) (2026)

## Abstract (arXiv)

Artificial Intelligence models encoding biology and chemistry are opening new routes to high-throughput and high-quality in-silico drug development. However, their training increasingly relies on computational scale, with recent protein language models (pLM) training on hundreds of graphical processing units (GPUs). We introduce the BioNeMo Framework to facilitate the training of computational biology and chemistry AI models across hundreds of GPUs. Its modular design allows the integration of individual components, such as data loaders, into existing workflows and is open to community contributions. We detail technical features of the BioNeMo Framework through use cases such as pLM pre-training and fine-tuning. On 256 NVIDIA A100s, BioNeMo Framework trains a three billion parameter BERT-based pLM on over one trillion tokens in 4.2 days. The BioNeMo Framework is open-source and free for everyone to use.