# CryoAI: Amortized Inference of Poses for Ab Initio Reconstruction of 3D Molecular Volumes from Real Cryo-EM Images

arXiv: https://arxiv.org/abs/2203.08138 · HTML: https://mlsanity.com/paper/2203.08138/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 73 references that are in the map.
- First posted on arXiv 15 Mar 2022.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)
- [Learning Shape Templates with Structured Implicit Functions](https://mlsanity.com/paper/1904.06447/) (2019)
- [Reconstructing continuous distributions of 3D protein structure from cryo-EM images](https://mlsanity.com/paper/1909.05215/) (2019)
- [Deep Generative Modeling for Volume Reconstruction in Cryo-Electron Microscopy](https://mlsanity.com/paper/2201.02867/) (2022)
- [Unsupervised Deep Learning Methods for Biological Image Reconstruction and Enhancement](https://mlsanity.com/paper/2105.08040/) (2021)
- [Inferring a Continuous Distribution of Atom Coordinates from Cryo-EM Images using VAEs](https://mlsanity.com/paper/2106.14108/) (2021)
- [Estimation of Orientation and Camera Parameters from Cryo-Electron Microscopy Images with Variational Autoencoders and Generative Adversarial Networks](https://mlsanity.com/paper/1911.08121/) (2019)
- [A common lines approach for ab-initio modeling of cyclically-symmetric molecules](https://mlsanity.com/paper/1901.10888/) (2019)
- [Differentiable probabilistic models of scientific imaging with the Fourier slice theorem](https://mlsanity.com/paper/1906.07582/) (2019)
- [Joint Angular Refinement and Reconstruction for Single-Particle Cryo-EM](https://mlsanity.com/paper/2003.10062/) (2020)

## Built on by (most cited first, 24 of 24)

- [Residual Multiplicative Filter Networks for Multiscale Reconstruction](https://mlsanity.com/paper/2206.00746/) (2022)
- [Amortized Inference for Heterogeneous Reconstruction in Cryo-EM](https://mlsanity.com/paper/2210.07387/) (2022)
- [Image to Sphere: Learning Equivariant Features for Efficient Pose Prediction](https://mlsanity.com/paper/2302.13926/) (2023)
- [MELON: NeRF with Unposed Images in SO(3)](https://mlsanity.com/paper/2303.08096/) (2023)
- [CryoGEM: Physics-Informed Generative Cryo-Electron Microscopy](https://mlsanity.com/paper/2312.02235/) (2023)
- [CryoSPIN: Improving Ab-Initio Cryo-EM Reconstruction with Semi-Amortized Pose Inference](https://mlsanity.com/paper/2406.10455/) (2024)
- [CryoFormer: Continuous Heterogeneous Cryo-EM Reconstruction using Transformer-based Neural Representations](https://mlsanity.com/paper/2303.16254/) (2023)
- [CryoFastAR: Fast Cryo-EM Ab Initio Reconstruction Made Easy](https://mlsanity.com/paper/2506.05864/) (2025)
- [Recovering a Molecule's 3D Dynamics from Liquid-phase Electron Microscopy Movies](https://mlsanity.com/paper/2308.11927/) (2023)
- [Heterogeneous reconstruction of deformable atomic models in Cryo-EM](https://mlsanity.com/paper/2209.15121/) (2022)
- [Boosted ab initio Cryo-EM 3D Reconstruction with ACE-EM](https://mlsanity.com/paper/2302.06091/) (2023)
- [Using VAEs to Learn Latent Variables: Observations on Applications in cryo-EM](https://mlsanity.com/paper/2303.07487/) (2023)
- [Improved Cryo-EM Pose Estimation and 3D Classification through Latent-Space Disentanglement](https://mlsanity.com/paper/2308.04956/) (2023)
- [Equivariant amortized inference of poses for cryo-EM](https://mlsanity.com/paper/2406.01630/) (2024)
- [cryoSPHERE: Single-particle heterogeneous reconstruction from cryo EM](https://mlsanity.com/paper/2407.01574/) (2024)
- [Deep Neural-network Prior for Orbit Recovery from Method of Moments](https://mlsanity.com/paper/2304.14604/) (2023)
- [CryoChains: Heterogeneous Reconstruction of Molecular Assembly of Semi-flexible Chains from Cryo-EM Images](https://mlsanity.com/paper/2306.07274/) (2023)
- [Cryo-forum: A framework for orientation recovery with uncertainty measure with the application in cryo-EM image analysis](https://mlsanity.com/paper/2307.09847/) (2023)
- [Augmenting x-ray single particle imaging reconstruction with self-supervised machine learning](https://mlsanity.com/paper/2311.16652/) (2023)
- [Scalable 3D Reconstruction From Single Particle X-Ray Diffraction Images Based on Online Machine Learning](https://mlsanity.com/paper/2312.14432/) (2023)
- [DualContrast: Unsupervised Disentangling of Content and Transformations with Implicit Parameterization](https://mlsanity.com/paper/2405.16796/) (2024)
- [CryoSplat: Gaussian Splatting for Cryo-EM Homogeneous Reconstruction](https://mlsanity.com/paper/2508.04929/) (2025)
- [Unsupervised SE(3) Disentanglement for in situ Macromolecular Morphology Identification from Cryo-Electron Tomography](https://mlsanity.com/paper/2601.01364/) (2026)
- [ARCHER: Amortized cross-specimen pose estimation for cryo-electron microscopy](https://mlsanity.com/paper/2608.22029/) (2026)

## Abstract (arXiv)

Cryo-electron microscopy (cryo-EM) has become a tool of fundamental importance in structural biology, helping us understand the basic building blocks of life. The algorithmic challenge of cryo-EM is to jointly estimate the unknown 3D poses and the 3D electron scattering potential of a biomolecule from millions of extremely noisy 2D images. Existing reconstruction algorithms, however, cannot easily keep pace with the rapidly growing size of cryo-EM datasets due to their high computational and memory cost. We introduce cryoAI, an ab initio reconstruction algorithm for homogeneous conformations that uses direct gradient-based optimization of particle poses and the electron scattering potential from single-particle cryo-EM data. CryoAI combines a learned encoder that predicts the poses of each particle image with a physics-based decoder to aggregate each particle image into an implicit representation of the scattering potential volume. This volume is stored in the Fourier domain for computational efficiency and leverages a modern coordinate network architecture for memory efficiency. Combined with a symmetrized loss function, this framework achieves results of a quality on par with state-of-the-art cryo-EM solvers for both simulated and experimental data, one order of magnitude faster for large datasets and with significantly lower memory requirements than existing methods.