# Protein structure generation via folding diffusion

arXiv: https://arxiv.org/abs/2209.15611 · HTML: https://mlsanity.com/paper/2209.15611/

## Key facts

- Cited by 115 papers in the map (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 22 of its 88 references that are in the map.
- First posted on arXiv 30 Sep 2022.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Equivariant Diffusion for Molecule Generation in 3D](https://mlsanity.com/paper/2203.17003/) (2022)
- [Torsional Diffusion for Molecular Conformer Generation](https://mlsanity.com/paper/2206.01729/) (2022)
- [Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem](https://mlsanity.com/paper/2206.04119/) (2022)
- [Efficient Large Scale Language Modeling with Mixtures of Experts](https://mlsanity.com/paper/2112.10684/) (2021)
- [Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2205.15019/) (2022)
- [CRASH: Raw Audio Score-based Generative Modeling for Controllable High-resolution Drum Sound Synthesis](https://mlsanity.com/paper/2106.07431/) (2021)

## Built on by (most cited first, 30 of 115)

- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [SE(3) diffusion model with application to protein backbone generation](https://mlsanity.com/paper/2302.02277/) (2023)
- [SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control](https://mlsanity.com/paper/2210.17432/) (2022)
- [SE(3)-Stochastic Flow Matching for Protein Backbone Generation](https://mlsanity.com/paper/2310.02391/) (2023)
- [MiDi: Mixed Graph and 3D Denoising Diffusion for Molecule Generation](https://mlsanity.com/paper/2302.09048/) (2023)
- [Diffusion Language Models Are Versatile Protein Learners](https://mlsanity.com/paper/2402.18567/) (2024)
- [Score-based Diffusion Models in Function Space](https://mlsanity.com/paper/2302.07400/) (2023)
- [Generative Diffusion Models on Graphs: Methods and Applications](https://mlsanity.com/paper/2302.02591/) (2023)
- [Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamics](https://mlsanity.com/paper/2302.00600/) (2023)
- [A Text-guided Protein Design Framework](https://mlsanity.com/paper/2302.04611/) (2023)
- [ProtAgents: Protein discovery via large language model multi-agent collaborations combining physics and machine learning](https://mlsanity.com/paper/2402.04268/) (2024)
- [DPLM-2: A Multimodal Diffusion Protein Language Model](https://mlsanity.com/paper/2410.13782/) (2024)
- [EigenFold: Generative Protein Structure Prediction with Diffusion Models](https://mlsanity.com/paper/2304.02198/) (2023)
- [Diffusion Models for Constrained Domains](https://mlsanity.com/paper/2304.05364/) (2023)
- [On conditional diffusion models for PDE simulations](https://mlsanity.com/paper/2410.16415/) (2024)
- [Generating Novel, Designable, and Diverse Protein Structures by Equivariantly Diffusing Oriented Residue Clouds](https://mlsanity.com/paper/2301.12485/) (2023)
- [Protein Conformation Generation via Force-Guided SE(3) Diffusion Models](https://mlsanity.com/paper/2403.14088/) (2024)
- [Graph Denoising Diffusion for Inverse Protein Folding](https://mlsanity.com/paper/2306.16819/) (2023)
- [Fast Graph Generation via Spectral Diffusion](https://mlsanity.com/paper/2211.08892/) (2022)
- [Sequence-Augmented SE(3)-Flow Matching For Conditional Protein Backbone Generation](https://mlsanity.com/paper/2405.20313/) (2024)
- [A Survey on Graph Diffusion Models: Generative AI in Science for Molecule, Protein and Material](https://mlsanity.com/paper/2304.01565/) (2023)
- [AbDiffuser: Full-Atom Generation of in vitro Functioning Antibodies](https://mlsanity.com/paper/2308.05027/) (2023)
- [Out of Many, One: Designing and Scaffolding Proteins at the Scale of the Structural Universe with Genie 2](https://mlsanity.com/paper/2405.15489/) (2024)
- [Full-Atom Peptide Design based on Multi-modal Flow Matching](https://mlsanity.com/paper/2406.00735/) (2024)
- [Str2Str: A Score-based Framework for Zero-shot Protein Conformation Sampling](https://mlsanity.com/paper/2306.03117/) (2023)
- [A Survey of Generative AI for de novo Drug Design: New Frontiers in Molecule and Protein Generation](https://mlsanity.com/paper/2402.08703/) (2024)
- [Faster Diffusion Sampling with Randomized Midpoints: Sequential and Parallel](https://mlsanity.com/paper/2406.00924/) (2024)
- [A Latent Diffusion Model for Protein Structure Generation](https://mlsanity.com/paper/2305.04120/) (2023)
- [Learning Joint 2D & 3D Diffusion Models for Complete Molecule Generation](https://mlsanity.com/paper/2305.12347/) (2023)
- [PPFlow: Target-aware Peptide Design with Torsional Flow Matching](https://mlsanity.com/paper/2405.06642/) (2024)

## Abstract (arXiv)

The ability to computationally generate novel yet physically foldable protein structures could lead to new biological discoveries and new treatments targeting yet incurable diseases. Despite recent advances in protein structure prediction, directly generating diverse, novel protein structures from neural networks remains difficult. In this work, we present a new diffusion-based generative model that designs protein backbone structures via a procedure that mirrors the native folding process. We describe protein backbone structure as a series of consecutive angles capturing the relative orientation of the constituent amino acid residues, and generate new structures by denoising from a random, unfolded state towards a stable folded structure. Not only does this mirror how proteins biologically twist into energetically favorable conformations, the inherent shift and rotational invariance of this representation crucially alleviates the need for complex equivariant networks. We train a denoising diffusion probabilistic model with a simple transformer backbone and demonstrate that our resulting model unconditionally generates highly realistic protein structures with complexity and structural patterns akin to those of naturally-occurring proteins. As a useful resource, we release the first open-source codebase and trained models for protein structure diffusion.