# Deep Generative Modeling for Protein Design

arXiv: https://arxiv.org/abs/2109.13754 · HTML: https://mlsanity.com/paper/2109.13754/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 96 references that are in the map.
- First posted on arXiv 31 Aug 2021.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [Residual Flows for Invertible Generative Modeling](https://mlsanity.com/paper/1906.02735/) (2019)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities](https://mlsanity.com/paper/2006.02425/) (2020)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing](https://mlsanity.com/paper/2007.06225/) (2020)
- [ProGen: Language Modeling for Protein Generation](https://mlsanity.com/paper/2004.03497/) (2020)
- [Conditioning by adaptive sampling for robust design](https://mlsanity.com/paper/1901.10060/) (2019)
- [Flow Contrastive Estimation of Energy-Based Models](https://mlsanity.com/paper/1912.00589/) (2019)
- [Protein sequence design with deep generative models](https://mlsanity.com/paper/2104.04457/) (2021)
- [Deep Learning in Protein Structural Modeling and Design](https://mlsanity.com/paper/2007.08383/) (2020)
- [Energy-based models for atomic-resolution protein conformations](https://mlsanity.com/paper/2004.13167/) (2020)

## Built on by (most cited first, 17 of 17)

- [Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2205.15019/) (2022)
- [PiFold: Toward effective and efficient protein inverse folding](https://mlsanity.com/paper/2209.12643/) (2022)
- [An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization](https://mlsanity.com/paper/2404.07771/) (2024)
- [Ultra-High-Resolution Detector Simulation with Intra-Event Aware GAN and Self-Supervised Relational Reasoning](https://mlsanity.com/paper/2303.08046/) (2023)
- [Mathematical analysis of singularities in the diffusion model under the submanifold assumption](https://mlsanity.com/paper/2301.07882/) (2023)
- [Knowledge-Design: Pushing the Limit of Protein Design via Knowledge Refinement](https://mlsanity.com/paper/2305.15151/) (2023)
- [DiffSDS: A language diffusion model for protein backbone inpainting under geometric conditions and constraints](https://mlsanity.com/paper/2301.09642/) (2023)
- [Steering Protein Language Models](https://mlsanity.com/paper/2509.07983/) (2025)
- [Generative artificial intelligence for de novo protein design](https://mlsanity.com/paper/2310.09685/) (2023)
- [Artificial intelligence-driven antimicrobial peptide discovery](https://mlsanity.com/paper/2308.10921/) (2023)
- [SpecMER: Fast Protein Generation with K-mer Guided Speculative Decoding](https://mlsanity.com/paper/2509.21689/) (2025)
- [Graph Representation Learning for Interactive Biomolecule Systems](https://mlsanity.com/paper/2304.02656/) (2023)
- [ConfHit: Conformal Generative Design with Oracle Free Guarantees](https://mlsanity.com/paper/2603.07371/) (2026)
- [AI driven B-cell Immunotherapy Design](https://mlsanity.com/paper/2309.01122/) (2023)
- [Description Generation using Variational Auto-Encoders for precursor microRNA](https://mlsanity.com/paper/2311.17970/) (2023)
- [Revealing Subtle Phenotypes in Small Microscopy Datasets Using Latent Diffusion Models](https://mlsanity.com/paper/2502.09665/) (2025)
- [LOCO-EPI: Leave-one-chromosome-out (LOCO) as a benchmarking paradigm for deep learning based prediction of enhancer-promoter interactions](https://mlsanity.com/paper/2504.00306/) (2025)

## Abstract (arXiv)

Deep learning approaches have produced substantial breakthroughs in fields such as image classification and natural language processing and are making rapid inroads in the area of protein design. Many generative models of proteins have been developed that encompass all known protein sequences, model specific protein families, or extrapolate the dynamics of individual proteins. Those generative models can learn protein representations that are often more informative of protein structure and function than hand-engineered features. Furthermore, they can be used to quickly propose millions of novel proteins that resemble the native counterparts in terms of expression level, stability, or other attributes. The protein design process can further be guided by discriminative oracles to select candidates with the highest probability of having the desired properties. In this review, we discuss five classes of generative models that have been most successful at modeling proteins and provide a framework for model guided protein design.