# denoiSplit: a method for joint microscopy image splitting and unsupervised denoising

arXiv: https://arxiv.org/abs/2403.11854 · HTML: https://mlsanity.com/paper/2403.11854/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 18 Mar 2024.

## Builds on (most cited first, 21 of 21)

- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Ladder Variational Autoencoders](https://mlsanity.com/paper/1602.02282/) (2016)
- [FFA-Net: Feature Fusion Attention Network for Single Image Dehazing](https://mlsanity.com/paper/1911.07559/) (2019)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- ["Double-DIP": Unsupervised Image Decomposition via Coupled Deep-Image-Priors](https://mlsanity.com/paper/1812.00467/) (2018)
- [Domain Adaptation for Image Dehazing](https://mlsanity.com/paper/2005.04668/) (2020)
- [Evaluating and Calibrating Uncertainty Prediction in Regression Tasks](https://mlsanity.com/paper/1905.11659/) (2019)
- [From Synthetic to Real: Image Dehazing Collaborating with Unlabeled Real Data](https://mlsanity.com/paper/2108.02934/) (2021)
- [Probabilistic Noise2Void: Unsupervised Content-Aware Denoising](https://mlsanity.com/paper/1906.00651/) (2019)
- [Interpretable Unsupervised Diversity Denoising and Artefact Removal](https://mlsanity.com/paper/2104.01374/) (2021)
- [Isotropic reconstruction of 3D fluorescence microscopy images using convolutional neural networks](https://mlsanity.com/paper/1704.01510/) (2017)
- [Fully Unsupervised Probabilistic Noise2Void](https://mlsanity.com/paper/1911.12291/) (2019)
- [Fully Unsupervised Diversity Denoising with Convolutional Variational Autoencoders](https://mlsanity.com/paper/2006.06072/) (2020)
- [μSplit: efficient image decomposition for microscopy data](https://mlsanity.com/paper/2211.12872/) (2022)
- [Self2Self+: Single-Image Denoising with Self-Supervised Learning and Image Quality Assessment Loss](https://mlsanity.com/paper/2307.10695/) (2023)

## Built on by (most cited first, 3 of 3)

- [MicroSSIM: Improved Structural Similarity for Comparing Microscopy Data](https://mlsanity.com/paper/2408.08747/) (2024)
- [scSplit: Bringing Severity Cognizance to Image Decomposition in Fluorescence Microscopy](https://mlsanity.com/paper/2503.22983/) (2025)
- [ResMatching: Noise-Resilient Computational Super-Resolution via Guided Conditional Flow Matching](https://mlsanity.com/paper/2510.26601/) (2025)

## Abstract (arXiv)

In this work, we present denoiSplit, a method to tackle a new analysis task, i.e. the challenge of joint semantic image splitting and unsupervised denoising. This dual approach has important applications in fluorescence microscopy, where semantic image splitting has important applications but noise does generally hinder the downstream analysis of image content. Image splitting involves dissecting an image into its distinguishable semantic structures. We show that the current state-of-the-art method for this task struggles in the presence of image noise, inadvertently also distributing the noise across the predicted outputs. The method we present here can deal with image noise by integrating an unsupervised denoising subtask. This integration results in improved semantic image unmixing, even in the presence of notable and realistic levels of imaging noise. A key innovation in denoiSplit is the use of specifically formulated noise models and the suitable adjustment of KL-divergence loss for the high-dimensional hierarchical latent space we are training. We showcase the performance of denoiSplit across multiple tasks on real-world microscopy images. Additionally, we perform qualitative and quantitative evaluations and compare the results to existing benchmarks, demonstrating the effectiveness of using denoiSplit: a single Variational Splitting Encoder-Decoder (VSE) Network using two suitable noise models to jointly perform semantic splitting and denoising.