# End-to-End Simultaneous Learning of Single-particle Orientation and 3D Map Reconstruction from Cryo-electron Microscopy Data

arXiv: https://arxiv.org/abs/2107.02958 · HTML: https://mlsanity.com/paper/2107.02958/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 45 references that are in the map.
- First posted on arXiv 7 Jul 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [A Review of Convolutional Neural Networks for Inverse Problems in Imaging](https://mlsanity.com/paper/1710.04011/) (2017)
- [Explorations in Homeomorphic Variational Auto-Encoding](https://mlsanity.com/paper/1807.04689/) (2018)
- [Inferring a Continuous Distribution of Atom Coordinates from Cryo-EM Images using VAEs](https://mlsanity.com/paper/2106.14108/) (2021)
- [Designing RNA Secondary Structures is Hard](https://mlsanity.com/paper/1710.11513/) (2017)
- [DeepPicker: a Deep Learning Approach for Fully Automated Particle Picking in Cryo-EM](https://mlsanity.com/paper/1605.01838/) (2016)
- [Estimation of Orientation and Camera Parameters from Cryo-Electron Microscopy Images with Variational Autoencoders and Generative Adversarial Networks](https://mlsanity.com/paper/1911.08121/) (2019)
- [A deep convolutional neural network approach to single-particle recognition in cryo-electron microscopy](https://mlsanity.com/paper/1605.05543/) (2016)
- [A common lines approach for ab-initio modeling of cyclically-symmetric molecules](https://mlsanity.com/paper/1901.10888/) (2019)
- [Differentiable probabilistic models of scientific imaging with the Fourier slice theorem](https://mlsanity.com/paper/1906.07582/) (2019)
- [Joint Angular Refinement and Reconstruction for Single-Particle Cryo-EM](https://mlsanity.com/paper/2003.10062/) (2020)

## Built on by (most cited first, 3 of 3)

- [Deep Generative Modeling for Volume Reconstruction in Cryo-Electron Microscopy](https://mlsanity.com/paper/2201.02867/) (2022)
- [Heterogeneous reconstruction of deformable atomic models in Cryo-EM](https://mlsanity.com/paper/2209.15121/) (2022)
- [Repeat and Concatenate: 2D to 3D Image Translation with 3D to 3D Generative Modeling](https://mlsanity.com/paper/2406.18422/) (2024)

## Abstract (arXiv)

Cryogenic electron microscopy (cryo-EM) provides images from different copies of the same biomolecule in arbitrary orientations. Here, we present an end-to-end unsupervised approach that learns individual particle orientations from cryo-EM data while reconstructing the average 3D map of the biomolecule, starting from a random initialization. The approach relies on an auto-encoder architecture where the latent space is explicitly interpreted as orientations used by the decoder to form an image according to the linear projection model. We evaluate our method on simulated data and show that it is able to reconstruct 3D particle maps from noisy- and CTF-corrupted 2D projection images of unknown particle orientations.