# Zero-Shot Image Denoising for High-Resolution Electron Microscopy

arXiv: https://arxiv.org/abs/2406.14264 · HTML: https://mlsanity.com/paper/2406.14264/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 54 references that are in the map.
- First posted on arXiv 20 Jun 2024.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [MemNet: A Persistent Memory Network for Image Restoration](https://mlsanity.com/paper/1708.02209/) (2017)
- [Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation](https://mlsanity.com/paper/2003.13659/) (2020)
- [High-Quality Self-Supervised Deep Image Denoising](https://mlsanity.com/paper/1901.10277/) (2019)
- [Noisier2Noise: Learning to Denoise from Unpaired Noisy Data](https://mlsanity.com/paper/1910.11908/) (2019)
- [AP-BSN: Self-Supervised Denoising for Real-World Images via Asymmetric PD and Blind-Spot Network](https://mlsanity.com/paper/2203.11799/) (2022)
- [Unpaired Learning of Deep Image Denoising](https://mlsanity.com/paper/2008.13711/) (2020)
- [Zero-Shot Noise2Noise: Efficient Image Denoising without any Data](https://mlsanity.com/paper/2303.11253/) (2023)
- [You Only Look Yourself: Unsupervised and Untrained Single Image Dehazing Neural Network](https://mlsanity.com/paper/2006.16829/) (2020)
- [Noise2Same: Optimizing A Self-Supervised Bound for Image Denoising](https://mlsanity.com/paper/2010.11971/) (2020)
- [When AWGN-based Denoiser Meets Real Noises](https://mlsanity.com/paper/1904.03485/) (2019)
- [Noise2Fast: Fast Self-Supervised Single Image Blind Denoising](https://mlsanity.com/paper/2108.10209/) (2021)
- [A Review Paper: Noise Models in Digital Image Processing](https://mlsanity.com/paper/1505.03489/) (2015)
- [FBI-Denoiser: Fast Blind Image Denoiser for Poisson-Gaussian Noise](https://mlsanity.com/paper/2105.10967/) (2021)
- [Review: Deep Learning in Electron Microscopy](https://mlsanity.com/paper/2009.08328/) (2020)
- [Fully Convolutional Pixel Adaptive Image Denoiser](https://mlsanity.com/paper/1807.07569/) (2018)
- [Deep Denoising For Scientific Discovery: A Case Study In Electron Microscopy](https://mlsanity.com/paper/2010.12970/) (2020)
- [PoGaIN: Poisson-Gaussian Image Noise Modeling from Paired Samples](https://mlsanity.com/paper/2210.04866/) (2022)
- [Noise2SR: Learning to Denoise from Super-Resolved Single Noisy Fluorescence Image](https://mlsanity.com/paper/2209.06411/) (2022)

## Abstract (arXiv)

High-resolution electron microscopy (HREM) imaging technique is a powerful tool for directly visualizing a broad range of materials in real-space. However, it faces challenges in denoising due to ultra-low signal-to-noise ratio (SNR) and scarce data availability. In this work, we propose Noise2SR, a zero-shot self-supervised learning (ZS-SSL) denoising framework for HREM. Within our framework, we propose a super-resolution (SR) based self-supervised training strategy, incorporating the Random Sub-sampler module. The Random Sub-sampler is designed to generate approximate infinite noisy pairs from a single noisy image, serving as an effective data augmentation in zero-shot denoising. Noise2SR trains the network with paired noisy images of different resolutions, which is conducted via SR strategy. The SR-based training facilitates the network adopting more pixels for supervision, and the random sub-sampling helps compel the network to learn continuous signals enhancing the robustness. Meanwhile, we mitigate the uncertainty caused by random-sampling by adopting minimum mean squared error (MMSE) estimation for the denoised results. With the distinctive integration of training strategy and proposed designs, Noise2SR can achieve superior denoising performance using a single noisy HREM image. We evaluate the performance of Noise2SR in both simulated and real HREM denoising tasks. It outperforms state-of-the-art ZS-SSL methods and achieves comparable denoising performance with supervised methods. The success of Noise2SR suggests its potential for improving the SNR of images in material imaging domains.