Latent Space Diffusion Models of Cryo-EM Structures
On the map: Diffusion models → Flow matching, time series and science
Abstract (arXiv)
Cryo-electron microscopy (cryo-EM) is unique among tools in structural biology in its ability to image large, dynamic protein complexes. Key to this ability is image processing algorithms for heterogeneous cryo-EM reconstruction, including recent deep learning-based approaches. The state-of-the-art method cryoDRGN uses a Variational Autoencoder (VAE) framework to learn a continuous distribution of protein structures from single particle cryo-EM imaging data. While cryoDRGN can model complex structural motions, the Gaussian prior distribution of the VAE fails to match the aggregate approximate posterior, which prevents generative sampling of structures especially for multi-modal distributions (e.g. compositional heterogeneity). Here, we train a diffusion model as an expressive, learnable prior in the cryoDRGN framework. Our approach learns a high-quality generative model over molecular conformations directly from cryo-EM imaging data. We show the ability to sample from the model on two synthetic and two real datasets, where samples accurately follow the data distribution unlike samples from the VAE prior distribution. We also demonstrate how the diffusion model prior can be leveraged for fast latent space traversal and interpolation between states of interest. By learning an accurate model of the data distribution, our method unlocks tools in generative modeling, sampling, and distribution analysis for heterogeneous cryo-EM ensembles.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Flow matching, time series and science, one of 14 directions in Diffusion models. Builds on 42 of its 67 references that are in the map. First posted on arXiv 25 Nov 2022.
Builds on
42 of 67 references.
Closest ancestors · 5
Same direction, newest first
- 2022Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem8 Jun
- 2022Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilistic Models26 May
- 2022Equivariant Diffusion for Molecule Generation in 3D31 Mar
- 2022GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation6 Mar
- 2022Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations5 Feb
Other references · 37
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2013Auto-Encoding Variational Bayescited by 4,053
- 2018Neural Ordinary Differential Equationscited by 3,779
- 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
- 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
25 more
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
- 2021Image Super-Resolution via Iterative Refinementcited by 1,511
- 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
- 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 637
- 2021Score-based Generative Modeling in Latent Spacecited by 636
- 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
- 2021Neural Fields in Visual Computing and Beyondcited by 540
- 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 516
- 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 449
- 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 264
- 2022Torsional Diffusion for Molecular Conformer Generationcited by 263
- 2017VAE with a VampPriorcited by 248
- 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusioncited by 233
- 2021Learning Gradient Fields for Molecular Conformation Generationcited by 163
- 2022Diffusion-based Molecule Generation with Informative Prior Bridgescited by 106
- 2020VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Modelscited by 104
- 2022DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latentscited by 104
- 2019NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transportcited by 74
- 2018Resampled Priors for Variational Autoencoderscited by 57
- 2019Reconstructing continuous distributions of 3D protein structure from cryo-EM imagescited by 56
- 2018Distribution Matching in Variational Inferencecited by 35
- 2018Variational Autoencoder with Implicit Optimal Priorscited by 33
- 2022Deep Generative Modeling for Volume Reconstruction in Cryo-Electron Microscopycited by 19
- 2021Inferring a Continuous Distribution of Atom Coordinates from Cryo-EM Images using VAEscited by 17
- 2019All SMILES Variational Autoencodercited by 15
Built on by
Citing papers per year: 2023: 2; 2024: 3; 2025: 1.
6 papers.
- 2023Boundary Guided Learning-Free Semantic Control with Diffusion Modelscited by 23
- 2025Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approachcited by 18
- 2023Diffusion Models in Bioinformatics: A New Wave of Deep Learning Revolution in Actioncited by 15
- 2024CryoFM: A Flow-based Foundation Model for Cryo-EM Densitiescited by 4
- 2024Diffusion-driven lensless fiber endomicroscopic quantitative phase imaging towards digital pathologycited by 0
- 2024Denoising diffusion models for high-resolution microscopy image restorationcited by 0
- 2025Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approach4 Jun 2025
- 2024CryoFM: A Flow-based Foundation Model for Cryo-EM Densities11 Oct 2024
- 2024Denoising diffusion models for high-resolution microscopy image restoration18 Sep 2024
- 2024Diffusion-driven lensless fiber endomicroscopic quantitative phase imaging towards digital pathology26 Jul 2024
- 2023Boundary Guided Learning-Free Semantic Control with Diffusion Models16 Feb 2023
- 2023Diffusion Models in Bioinformatics: A New Wave of Deep Learning Revolution in Action13 Feb 2023
Similar, not cited
Close in wording, no citation either way
- 2026Cryo-SWAN: the Multi-Scale Wavelet-decomposition-inspired Autoencoder Network for molecular density representation of molecular volumes
- 2025CryoCCD: Conditional Cycle-consistent Diffusion with Biophysical Modeling for Cryo-EM Synthesis
- 2025cryoSENSE: Compressive Sensing Enables High-throughput Microscopy with Sparse and Generative Priors on the Protein Cryo-EM Image Manifold
- 2026CryoNet.Refine: A One-step Diffusion Model for Rapid Refinement of Structural Models with Cryo-EM Density Map Restraints
- 2026Latent Diffusion Pretraining for Crystal Property Prediction
- 2025SemiETPicker: Fast and Label-Efficient Particle Picking for CryoET Tomography Using Semi-Supervised Learning
- 2024Robust single-particle cryo-EM image denoising and restoration
- 2025Compressing Biology: Evaluating the Stable Diffusion VAE for Phenotypic Drug Discovery