# Joint Denoising of Cryo-EM Projection Images using Polar Transformers

arXiv: https://arxiv.org/abs/2506.11283 · HTML: https://mlsanity.com/paper/2506.11283/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 64 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network](https://mlsanity.com/paper/1806.09231/) (2018)
- [Amortized Inference for Heterogeneous Reconstruction in Cryo-EM](https://mlsanity.com/paper/2210.07387/) (2022)
- [Rotationally Invariant Image Representation for Viewing Direction Classification in Cryo-EM](https://mlsanity.com/paper/1309.7643/) (2013)
- [DeepPicker: a Deep Learning Approach for Fully Automated Particle Picking in Cryo-EM](https://mlsanity.com/paper/1605.01838/) (2016)
- [Cryo-CARE: Content-Aware Image Restoration for Cryo-Transmission Electron Microscopy Data](https://mlsanity.com/paper/1810.05420/) (2018)
- [cuFINUFFT: a load-balanced GPU library for general-purpose nonuniform FFTs](https://mlsanity.com/paper/2102.08463/) (2021)
- [Denoising and Covariance Estimation of Single Particle Cryo-EM Images](https://mlsanity.com/paper/1602.06632/) (2016)
- [Unsupervised Object Representation Learning using Translation and Rotation Group Equivariant VAE](https://mlsanity.com/paper/2210.12918/) (2022)
- [Learning Rotation Invariant Features for Cryogenic Electron Microscopy Image Reconstruction](https://mlsanity.com/paper/2101.03549/) (2021)
- [Rotation and Translation Invariant Representation Learning with Implicit Neural Representations](https://mlsanity.com/paper/2304.13995/) (2023)

## Abstract (arXiv)

Many imaging modalities involve reconstruction of unknown objects from collections of noisy projections related by random rotations. In one of these modalities, cryogenic electron microscopy (cryo-EM), the extremely low signal-to-noise ratio (SNR) makes integration of information from multiple images crucial. Existing approaches to cryo-EM processing, however, either rely on handcrafted priors or apply deep learning only on select portions of the pipeline, such as particle picking, micrograph denoising, or refinement. A fully end-to-end reconstruction approach requires a neural network architecture that integrates information from multiple images while respecting the rotational symmetry of the measurement process. In this work, we introduce the polar transformer, a new neural network architecture that combines polar representations and transformers along with a convolutional attention mechanism that preserves the rotational symmetry of the problem. We apply it to the particle-level denoising problem, where it is able to learn discriminative features in the images, enabling optimal clustering, alignment, and denoising. On simulated datasets, this achieves up to a 2× reduction in mean squared error (MSE) at a signal-to-noise ratio (SNR) of 0.02, suggesting new opportunities for data-driven approaches to reconstruction in cryo-EM and related tomographic modalities.