# Masked Pre-training Enables Universal Zero-shot Denoiser

arXiv: https://arxiv.org/abs/2401.14966 · HTML: https://mlsanity.com/paper/2401.14966/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 73 references that are in the map.
- First posted on arXiv 26 Jan 2024.

## Builds on (most cited first, 30 of 57)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [Benchmarking Denoising Algorithms with Real Photographs](https://mlsanity.com/paper/1707.01313/) (2017)
- [Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images](https://mlsanity.com/paper/2101.02824/) (2021)
- [High-Quality Self-Supervised Deep Image Denoising](https://mlsanity.com/paper/1901.10277/) (2019)
- [Noisier2Noise: Learning to Denoise from Unpaired Noisy Data](https://mlsanity.com/paper/1910.11908/) (2019)
- [Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks](https://mlsanity.com/paper/1810.03982/) (2018)
- [Variational Denoising Network: Toward Blind Noise Modeling and Removal](https://mlsanity.com/paper/1908.11314/) (2019)
- [Noise2Score: Tweedie's Approach to Self-Supervised Image Denoising without Clean Images](https://mlsanity.com/paper/2106.07009/) (2021)
- [Noise Flow: Noise Modeling with Conditional Normalizing Flows](https://mlsanity.com/paper/1908.08453/) (2019)
- [AP-BSN: Self-Supervised Denoising for Real-World Images via Asymmetric PD and Blind-Spot Network](https://mlsanity.com/paper/2203.11799/) (2022)
- [Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots](https://mlsanity.com/paper/2203.06967/) (2022)
- [Real-world Noisy Image Denoising: A New Benchmark](https://mlsanity.com/paper/1804.02603/) (2018)
- [Unpaired Learning of Deep Image Denoising](https://mlsanity.com/paper/2008.13711/) (2020)
- [A Poisson-Gaussian Denoising Dataset with Real Fluorescence Microscopy Images](https://mlsanity.com/paper/1812.10366/) (2018)

## Built on by (most cited first, 3 of 3)

- [FM2S: Towards Spatially-Correlated Noise Modeling in Zero-Shot Fluorescence Microscopy Image Denoising](https://mlsanity.com/paper/2412.10031/) (2024)
- [Zero-shot Denoising via Neural Compression: Theoretical and algorithmic framework](https://mlsanity.com/paper/2506.12693/) (2025)
- [Self-supervised Dynamic Heterogeneous Degradation Modeling for Unified Zero-Shot Image Restoration](https://mlsanity.com/paper/2605.24593/) (2026)

## Abstract (arXiv)

In this work, we observe that model trained on vast general images via masking strategy, has been naturally embedded with their distribution knowledge, thus spontaneously attains the underlying potential for strong image denoising. Based on this observation, we propose a novel zero-shot denoising paradigm, i.e., Masked Pre-train then Iterative fill (MPI). MPI first trains model via masking and then employs pre-trained weight for high-quality zero-shot image denoising on a single noisy image. Concretely, MPI comprises two key procedures: 1) Masked Pre-training involves training model to reconstruct massive natural images with random masking for generalizable representations, gathering the potential for valid zero-shot denoising on images with varying noise degradation and even in distinct image types. 2) Iterative filling exploits pre-trained knowledge for effective zero-shot denoising. It iteratively optimizes the image by leveraging pre-trained weights, focusing on alternate reconstruction of different image parts, and gradually assembles fully denoised image within limited number of iterations. Comprehensive experiments across various noisy scenarios underscore the notable advances of MPI over previous approaches with a marked reduction in inference time. Code available at https://github.com/krennic999/MPI.