# Interpretable Unsupervised Diversity Denoising and Artefact Removal

arXiv: https://arxiv.org/abs/2104.01374 · HTML: https://mlsanity.com/paper/2104.01374/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 55 references that are in the map.
- First posted on arXiv 3 Apr 2021.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [Deep Learning for Classical Japanese Literature](https://mlsanity.com/paper/1812.01718/) (2018)
- [PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications](https://mlsanity.com/paper/1701.05517/) (2017)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Ladder Variational Autoencoders](https://mlsanity.com/paper/1602.02282/) (2016)
- [The Little Engine that Could: Regularization by Denoising (RED)](https://mlsanity.com/paper/1611.02862/) (2016)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images](https://mlsanity.com/paper/2011.10650/) (2020)
- [Variational Lossy Autoencoder](https://mlsanity.com/paper/1611.02731/) (2016)
- [High-Quality Self-Supervised Deep Image Denoising](https://mlsanity.com/paper/1901.10277/) (2019)
- [BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling](https://mlsanity.com/paper/1902.02102/) (2019)
- [A Poisson-Gaussian Denoising Dataset with Real Fluorescence Microscopy Images](https://mlsanity.com/paper/1812.10366/) (2018)
- [Noise2Same: Optimizing A Self-Supervised Bound for Image Denoising](https://mlsanity.com/paper/2010.11971/) (2020)
- [Probabilistic Noise2Void: Unsupervised Content-Aware Denoising](https://mlsanity.com/paper/1906.00651/) (2019)
- [Structured Uncertainty Prediction Networks](https://mlsanity.com/paper/1802.07079/) (2018)
- [High Perceptual Quality Image Denoising with a Posterior Sampling CGAN](https://mlsanity.com/paper/2103.04192/) (2021)
- [Projected Distribution Loss for Image Enhancement](https://mlsanity.com/paper/2012.09289/) (2020)
- [DenoiSeg: Joint Denoising and Segmentation](https://mlsanity.com/paper/2005.02987/) (2020)
- [Isotropic reconstruction of 3D fluorescence microscopy images using convolutional neural networks](https://mlsanity.com/paper/1704.01510/) (2017)

## Built on by (most cited first, 26 of 26)

- [SPIRE: Semantic Prompt-Driven Image Restoration](https://mlsanity.com/paper/2312.11595/) (2023)
- [Score Priors Guided Deep Variational Inference for Unsupervised Real-World Single Image Denoising](https://mlsanity.com/paper/2308.04682/) (2023)
- [MicroSSIM: Improved Structural Similarity for Comparing Microscopy Data](https://mlsanity.com/paper/2408.08747/) (2024)
- [N2V2 -- Fixing Noise2Void Checkerboard Artifacts with Modified Sampling Strategies and a Tweaked Network Architecture](https://mlsanity.com/paper/2211.08512/) (2022)
- [Inverse problem regularization with hierarchical variational autoencoders](https://mlsanity.com/paper/2303.11217/) (2023)
- [Patch-Craft Self-Supervised Training for Correlated Image Denoising](https://mlsanity.com/paper/2211.09919/) (2022)
- [μSplit: efficient image decomposition for microscopy data](https://mlsanity.com/paper/2211.12872/) (2022)
- [Self-Supervised Learning from Noisy and Incomplete Data](https://mlsanity.com/paper/2601.03244/) (2026)
- [Evaluating Unsupervised Denoising Requires Unsupervised Metrics](https://mlsanity.com/paper/2210.05553/) (2022)
- [Image Denoising and the Generative Accumulation of Photons](https://mlsanity.com/paper/2307.06607/) (2023)
- [Learned, uncertainty-driven adaptive acquisition for photon-efficient scanning microscopy](https://mlsanity.com/paper/2310.16102/) (2023)
- [denoiSplit: a method for joint microscopy image splitting and unsupervised denoising](https://mlsanity.com/paper/2403.11854/) (2024)
- [HazeMatching: Dehazing Light Microscopy Images with Guided Conditional Flow Matching](https://mlsanity.com/paper/2506.22397/) (2025)
- [MMV_Im2Im: An Open Source Microscopy Machine Vision Toolbox for Image-to-Image Transformation](https://mlsanity.com/paper/2209.02498/) (2022)
- [Filter2Noise: A Framework for Interpretable and Zero-Shot Low-Dose CT Image Denoising](https://mlsanity.com/paper/2504.13519/) (2025)
- [Unsupervised Denoising for Signal-Dependent and Row-Correlated Imaging Noise](https://mlsanity.com/paper/2310.07887/) (2023)
- [scSplit: Bringing Severity Cognizance to Image Decomposition in Fluorescence Microscopy](https://mlsanity.com/paper/2503.22983/) (2025)
- [ResMatching: Noise-Resilient Computational Super-Resolution via Guided Conditional Flow Matching](https://mlsanity.com/paper/2510.26601/) (2025)
- [λSplit: Self-Supervised Content-Aware Spectral Unmixing for Fluorescence Microscopy](https://mlsanity.com/paper/2603.23647/) (2026)
- [Shot Noise Reduction in Radiographic and Tomographic Multi-Channel Imaging with Self-Supervised Deep Learning](https://mlsanity.com/paper/2303.14429/) (2023)
- [DeepContrast: Deep Tissue Contrast Enhancement using Synthetic Data Degradations and OOD Model Predictions](https://mlsanity.com/paper/2308.08365/) (2023)
- [Direct Unsupervised Denoising](https://mlsanity.com/paper/2310.18116/) (2023)
- [Diffusion Priors for Variational Likelihood Estimation and Image Denoising](https://mlsanity.com/paper/2410.17521/) (2024)
- [Hybrid Swin Attention Networks for Simultaneously Low-Dose PET and CT Denoising](https://mlsanity.com/paper/2509.06591/) (2025)
- [ε-Seg: Sparsely Supervised Semantic Segmentation of Microscopy Data](https://mlsanity.com/paper/2510.18637/) (2025)
- [SWITi: Quantifying and Reducing Tiling Artifacts with Sliding Window Inner Tiling](https://mlsanity.com/paper/2607.18990/) (2026)

## Abstract (arXiv)

Image denoising and artefact removal are complex inverse problems admitting multiple valid solutions. Unsupervised diversity restoration, that is, obtaining a diverse set of possible restorations given a corrupted image, is important for ambiguity removal in many applications such as microscopy where paired data for supervised training are often unobtainable. In real world applications, imaging noise and artefacts are typically hard to model, leading to unsatisfactory performance of existing unsupervised approaches. This work presents an interpretable approach for unsupervised and diverse image restoration. To this end, we introduce a capable architecture called Hierarchical DivNoising (HDN) based on hierarchical Variational Autoencoder. We show that HDN learns an interpretable multi-scale representation of artefacts and we leverage this interpretability to remove imaging artefacts commonly occurring in microscopy data. Our method achieves state-of-the-art results on twelve benchmark image denoising datasets while providing access to a whole distribution of sensibly restored solutions. Additionally, we demonstrate on three real microscopy datasets that HDN removes artefacts without supervision, being the first method capable of doing so while generating multiple plausible restorations all consistent with the given corrupted image.