# Proteina: Scaling Flow-based Protein Structure Generative Models

arXiv: https://arxiv.org/abs/2503.00710 · HTML: https://mlsanity.com/paper/2503.00710/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 80 references that are in the map.
- First posted on arXiv 2 Mar 2025.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [Improving and generalizing flow-based generative models with minibatch optimal transport](https://mlsanity.com/paper/2302.00482/) (2023)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [Scaling Vision Transformers to 22 Billion Parameters](https://mlsanity.com/paper/2302.05442/) (2023)
- [Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design](https://mlsanity.com/paper/2402.04997/) (2024)
- [Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation](https://mlsanity.com/paper/2303.00848/) (2023)
- [Guiding a Diffusion Model with a Bad Version of Itself](https://mlsanity.com/paper/2406.02507/) (2024)
- [Multisample Flow Matching: Straightening Flows with Minibatch Couplings](https://mlsanity.com/paper/2304.14772/) (2023)
- [Flow Matching on General Geometries](https://mlsanity.com/paper/2302.03660/) (2023)

## Built on by (most cited first, 30 of 65)

- [Reasoning with Sampling: Your Base Model is Smarter Than You Think](https://mlsanity.com/paper/2510.14901/) (2025)
- [La-Proteina: Atomistic Protein Generation via Partially Latent Flow Matching](https://mlsanity.com/paper/2507.09466/) (2025)
- [Tilt Matching for Scalable Sampling and Fine-Tuning](https://mlsanity.com/paper/2512.21829/) (2025)
- [SimpleFold: Folding Proteins is Simpler than You Think](https://mlsanity.com/paper/2509.18480/) (2025)
- [Flow Matching Meets Biology and Life Science: A Survey](https://mlsanity.com/paper/2507.17731/) (2025)
- [GARF: Learning Generalizable 3D Reassembly for Real-World Fractures](https://mlsanity.com/paper/2504.05400/) (2025)
- [TABASCO: A Fast, Simplified Model for Molecular Generation with Improved Physical Quality](https://mlsanity.com/paper/2507.00899/) (2025)
- [Amortized Sampling with Transferable Normalizing Flows](https://mlsanity.com/paper/2508.18175/) (2025)
- [Elucidating the Design Space of Multimodal Protein Language Models](https://mlsanity.com/paper/2504.11454/) (2025)
- [Flexible MOF Generation with Torsion-Aware Flow Matching](https://mlsanity.com/paper/2505.17914/) (2025)
- [Efficient Regression-Based Training of Normalizing Flows for Boltzmann Generators](https://mlsanity.com/paper/2506.01158/) (2025)
- [Score Distillation of Flow Matching Models](https://mlsanity.com/paper/2509.25127/) (2025)
- [Riemannian MeanFlow](https://mlsanity.com/paper/2602.07744/) (2026)
- [Temporal Score Rescaling for Temperature Sampling in Diffusion and Flow Models](https://mlsanity.com/paper/2510.01184/) (2025)
- [Sublinear iterations can suffice even for DDPMs](https://mlsanity.com/paper/2511.04844/) (2025)
- [Sampling 3D Molecular Conformers with Diffusion Transformers](https://mlsanity.com/paper/2506.15378/) (2025)
- [SafeProtein: Red-Teaming Framework and Benchmark for Protein Foundation Models](https://mlsanity.com/paper/2509.03487/) (2025)
- [UniSite: The First Cross-Structure Dataset and Learning Framework for End-to-End Ligand Binding Site Detection](https://mlsanity.com/paper/2506.03237/) (2025)
- [Efficient Molecular Conformer Generation with SO(3)-Averaged Flow Matching and Reflow](https://mlsanity.com/paper/2507.09785/) (2025)
- [Weighted Conditional Flow Matching](https://mlsanity.com/paper/2507.22270/) (2025)
- [Flow Autoencoders are Effective Protein Tokenizers](https://mlsanity.com/paper/2510.00351/) (2025)
- [Triangle Multiplication Is All You Need For Biomolecular Structure Representations](https://mlsanity.com/paper/2510.18870/) (2025)
- [Atomic Diffusion Models for Small Molecule Structure Elucidation from NMR Spectra](https://mlsanity.com/paper/2512.03127/) (2025)
- [HD-Prot: A Protein Language Model for Joint Sequence-Structure Modeling with Continuous Structure Tokens](https://mlsanity.com/paper/2512.15133/) (2025)
- [Protein Language Model Embeddings Improve Generalization of Implicit Transfer Operators](https://mlsanity.com/paper/2602.11216/) (2026)
- [ProteinAE: Protein Diffusion Autoencoders for Structure Encoding](https://mlsanity.com/paper/2510.10634/) (2025)
- [Unified all-atom molecule generation with neural fields](https://mlsanity.com/paper/2511.15906/) (2025)
- [Fast Organic Crystal Structure Prediction with Unit Cell Flow Matching](https://mlsanity.com/paper/2606.03199/) (2026)
- [Generative Modeling of Full-Atom Protein Conformations using Latent Diffusion on Graph Embeddings](https://mlsanity.com/paper/2506.17064/) (2025)
- [Monte Carlo Tree Diffusion with Multiple Experts for Protein Design](https://mlsanity.com/paper/2509.15796/) (2025)

## Abstract (arXiv)

Recently, diffusion- and flow-based generative models of protein structures have emerged as a powerful tool for de novo protein design. Here, we develop Proteina, a new large-scale flow-based protein backbone generator that utilizes hierarchical fold class labels for conditioning and relies on a tailored scalable transformer architecture with up to 5x as many parameters as previous models. To meaningfully quantify performance, we introduce a new set of metrics that directly measure the distributional similarity of generated proteins with reference sets, complementing existing metrics. We further explore scaling training data to millions of synthetic protein structures and explore improved training and sampling recipes adapted to protein backbone generation. This includes fine-tuning strategies like LoRA for protein backbones, new guidance methods like classifier-free guidance and autoguidance for protein backbones, and new adjusted training objectives. Proteina achieves state-of-the-art performance on de novo protein backbone design and produces diverse and designable proteins at unprecedented length, up to 800 residues. The hierarchical conditioning offers novel control, enabling high-level secondary-structure guidance as well as low-level fold-specific generation.