# Amortized Inference for Heterogeneous Reconstruction in Cryo-EM

arXiv: https://arxiv.org/abs/2210.07387 · HTML: https://mlsanity.com/paper/2210.07387/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 45 references that are in the map.
- First posted on arXiv 13 Oct 2022.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Reconstructing continuous distributions of 3D protein structure from cryo-EM images](https://mlsanity.com/paper/1909.05215/) (2019)
- [Explicitly disentangling image content from translation and rotation with spatial-VAE](https://mlsanity.com/paper/1909.11663/) (2019)
- [CryoAI: Amortized Inference of Poses for Ab Initio Reconstruction of 3D Molecular Volumes from Real Cryo-EM Images](https://mlsanity.com/paper/2203.08138/) (2022)
- [Deep Generative Modeling for Volume Reconstruction in Cryo-Electron Microscopy](https://mlsanity.com/paper/2201.02867/) (2022)
- [Inferring a Continuous Distribution of Atom Coordinates from Cryo-EM Images using VAEs](https://mlsanity.com/paper/2106.14108/) (2021)
- [Estimation of Orientation and Camera Parameters from Cryo-Electron Microscopy Images with Variational Autoencoders and Generative Adversarial Networks](https://mlsanity.com/paper/1911.08121/) (2019)
- [Hyper-Molecules: on the Representation and Recovery of Dynamical Structures, with Application to Flexible Macro-Molecular Structures in Cryo-EM](https://mlsanity.com/paper/1907.01589/) (2019)

## Built on by (most cited first, 23 of 23)

- [Variational Bayesian Imaging with an Efficient Surrogate Score-based Prior](https://mlsanity.com/paper/2309.01949/) (2023)
- [CryoBench: Diverse and challenging datasets for the heterogeneity problem in cryo-EM](https://mlsanity.com/paper/2408.05526/) (2024)
- [Capturing dynamical correlations using implicit neural representations](https://mlsanity.com/paper/2304.03949/) (2023)
- [MELON: NeRF with Unposed Images in SO(3)](https://mlsanity.com/paper/2303.08096/) (2023)
- [CryoGEM: Physics-Informed Generative Cryo-Electron Microscopy](https://mlsanity.com/paper/2312.02235/) (2023)
- [CryoSPIN: Improving Ab-Initio Cryo-EM Reconstruction with Semi-Amortized Pose Inference](https://mlsanity.com/paper/2406.10455/) (2024)
- [CryoFormer: Continuous Heterogeneous Cryo-EM Reconstruction using Transformer-based Neural Representations](https://mlsanity.com/paper/2303.16254/) (2023)
- [CryoFastAR: Fast Cryo-EM Ab Initio Reconstruction Made Easy](https://mlsanity.com/paper/2506.05864/) (2025)
- [Recovering a Molecule's 3D Dynamics from Liquid-phase Electron Microscopy Movies](https://mlsanity.com/paper/2308.11927/) (2023)
- [Mixture of neural fields for heterogeneous reconstruction in cryo-EM](https://mlsanity.com/paper/2412.09420/) (2024)
- [Using VAEs to Learn Latent Variables: Observations on Applications in cryo-EM](https://mlsanity.com/paper/2303.07487/) (2023)
- [Improved Cryo-EM Pose Estimation and 3D Classification through Latent-Space Disentanglement](https://mlsanity.com/paper/2308.04956/) (2023)
- [Equivariant amortized inference of poses for cryo-EM](https://mlsanity.com/paper/2406.01630/) (2024)
- [CryoCCD: Conditional Cycle-consistent Diffusion with Biophysical Modeling for Cryo-EM Synthesis](https://mlsanity.com/paper/2505.23444/) (2025)
- [Reconstructing Heterogeneous Biomolecules via Hierarchical Gaussian Mixtures and Part Discovery](https://mlsanity.com/paper/2506.09063/) (2025)
- [CryoHype: Reconstructing a thousand cryo-EM structures with transformer-based hypernetworks](https://mlsanity.com/paper/2512.06332/) (2025)
- [Cryo-forum: A framework for orientation recovery with uncertainty measure with the application in cryo-EM image analysis](https://mlsanity.com/paper/2307.09847/) (2023)
- [Scalable 3D Reconstruction From Single Particle X-Ray Diffraction Images Based on Online Machine Learning](https://mlsanity.com/paper/2312.14432/) (2023)
- [DualContrast: Unsupervised Disentangling of Content and Transformations with Implicit Parameterization](https://mlsanity.com/paper/2405.16796/) (2024)
- [Joint Denoising of Cryo-EM Projection Images using Polar Transformers](https://mlsanity.com/paper/2506.11283/) (2025)
- [CryoSplat: Gaussian Splatting for Cryo-EM Homogeneous Reconstruction](https://mlsanity.com/paper/2508.04929/) (2025)
- [Protein Graph Neural Networks for Heterogeneous Cryo-EM Reconstruction](https://mlsanity.com/paper/2602.21915/) (2026)
- [ARCHER: Amortized cross-specimen pose estimation for cryo-electron microscopy](https://mlsanity.com/paper/2608.22029/) (2026)

## Abstract (arXiv)

Cryo-electron microscopy (cryo-EM) is an imaging modality that provides unique insights into the dynamics of proteins and other building blocks of life. The algorithmic challenge of jointly estimating the poses, 3D structure, and conformational heterogeneity of a biomolecule from millions of noisy and randomly oriented 2D projections in a computationally efficient manner, however, remains unsolved. Our method, cryoFIRE, performs ab initio heterogeneous reconstruction with unknown poses in an amortized framework, thereby avoiding the computationally expensive step of pose search while enabling the analysis of conformational heterogeneity. Poses and conformation are jointly estimated by an encoder while a physics-based decoder aggregates the images into an implicit neural representation of the conformational space. We show that our method can provide one order of magnitude speedup on datasets containing millions of images without any loss of accuracy. We validate that the joint estimation of poses and conformations can be amortized over the size of the dataset. For the first time, we prove that an amortized method can extract interpretable dynamic information from experimental datasets.