# OpenProteinSet: Training data for structural biology at scale

arXiv: https://arxiv.org/abs/2308.05326 · HTML: https://mlsanity.com/paper/2308.05326/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 85 references that are in the map.
- First posted on arXiv 10 Aug 2023.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [A Generalist Agent](https://mlsanity.com/paper/2205.06175/) (2022)
- [OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework](https://mlsanity.com/paper/2202.03052/) (2022)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound](https://mlsanity.com/paper/2201.02639/) (2022)
- [Translation between Molecules and Natural Language](https://mlsanity.com/paper/2204.11817/) (2022)
- [Pretrained Transformers as Universal Computation Engines](https://mlsanity.com/paper/2103.05247/) (2021)
- [Scaling Laws for Generative Mixed-Modal Language Models](https://mlsanity.com/paper/2301.03728/) (2023)
- [ProteinNet: a standardized data set for machine learning of protein structure](https://mlsanity.com/paper/1902.00249/) (2019)
- [Accurate De Novo Prediction of Protein Contact Map by Ultra-Deep Learning Model](https://mlsanity.com/paper/1609.00680/) (2016)
- [Generative power of a protein language model trained on multiple sequence alignments](https://mlsanity.com/paper/2204.07110/) (2022)
- [PSP: Million-level Protein Sequence Dataset for Protein Structure Prediction](https://mlsanity.com/paper/2206.12240/) (2022)
- [Multimodal Model with Text and Drug Embeddings for Adverse Drug Reaction Classification](https://mlsanity.com/paper/2210.13238/) (2022)

## Built on by (most cited first, 8 of 8)

- [AlphaFold Meets Flow Matching for Generating Protein Ensembles](https://mlsanity.com/paper/2402.04845/) (2024)
- [Bio-xLSTM: Generative modeling, representation and in-context learning of biological and chemical sequences](https://mlsanity.com/paper/2411.04165/) (2024)
- [MSAGPT: Neural Prompting Protein Structure Prediction via MSA Generative Pre-Training](https://mlsanity.com/paper/2406.05347/) (2024)
- [Computational Protein Science in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2501.10282/) (2025)
- [OneProt: Towards Multi-Modal Protein Foundation Models](https://mlsanity.com/paper/2411.04863/) (2024)
- [Lightweight MSA Design Advances Protein Folding From Evolutionary Embeddings](https://mlsanity.com/paper/2507.07032/) (2025)
- [Protein Folding with Neural Ordinary Differential Equations](https://mlsanity.com/paper/2510.16253/) (2025)
- [Evolution-Aware MSA Reasoning for Subsampling via Factor Graphs](https://mlsanity.com/paper/2607.22314/) (2026)

## Abstract (arXiv)

Multiple sequence alignments (MSAs) of proteins encode rich biological information and have been workhorses in bioinformatic methods for tasks like protein design and protein structure prediction for decades. Recent breakthroughs like AlphaFold2 that use transformers to attend directly over large quantities of raw MSAs have reaffirmed their importance. Generation of MSAs is highly computationally intensive, however, and no datasets comparable to those used to train AlphaFold2 have been made available to the research community, hindering progress in machine learning for proteins. To remedy this problem, we introduce OpenProteinSet, an open-source corpus of more than 16 million MSAs, associated structural homologs from the Protein Data Bank, and AlphaFold2 protein structure predictions. We have previously demonstrated the utility of OpenProteinSet by successfully retraining AlphaFold2 on it. We expect OpenProteinSet to be broadly useful as training and validation data for 1) diverse tasks focused on protein structure, function, and design and 2) large-scale multimodal machine learning research.