# DenoiSeg: Joint Denoising and Segmentation

arXiv: https://arxiv.org/abs/2005.02987 · HTML: https://mlsanity.com/paper/2005.02987/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 May 2020.

## Built on by (most cited first, 20 of 20)

- [Inverse-Dirichlet Weighting Enables Reliable Training of Physics Informed Neural Networks](https://mlsanity.com/paper/2107.00940/) (2021)
- [Interpretable Unsupervised Diversity Denoising and Artefact Removal](https://mlsanity.com/paper/2104.01374/) (2021)
- [From Denoising Training to Test-Time Adaptation: Enhancing Domain Generalization for Medical Image Segmentation](https://mlsanity.com/paper/2310.20271/) (2023)
- [CellTranspose: Few-shot Domain Adaptation for Cellular Instance Segmentation](https://mlsanity.com/paper/2212.14121/) (2022)
- [N2V2 -- Fixing Noise2Void Checkerboard Artifacts with Modified Sampling Strategies and a Tweaked Network Architecture](https://mlsanity.com/paper/2211.08512/) (2022)
- [μSplit: efficient image decomposition for microscopy data](https://mlsanity.com/paper/2211.12872/) (2022)
- [Synergy Between Semantic Segmentation and Image Denoising via Alternate Boosting](https://mlsanity.com/paper/2102.12095/) (2021)
- [Deep learning based domain adaptation for mitochondria segmentation on EM volumes](https://mlsanity.com/paper/2202.10773/) (2022)
- [Seeing Far in the Dark with Patterned Flash](https://mlsanity.com/paper/2207.12570/) (2022)
- [Self2Seg: Single-Image Self-Supervised Joint Segmentation and Denoising](https://mlsanity.com/paper/2309.10511/) (2023)
- [Zero-shot Denoising via Neural Compression: Theoretical and algorithmic framework](https://mlsanity.com/paper/2506.12693/) (2025)
- [FReSCO: Flow Reconstruction and Segmentation for low latency Cardiac Output monitoring using deep artifact suppression and segmentation](https://mlsanity.com/paper/2203.13729/) (2022)
- [Domino Denoise: An Accurate Blind Zero-Shot Denoiser using Domino Tilings](https://mlsanity.com/paper/2212.02439/) (2022)
- [MIGA: Mutual Information-Guided Attack on Denoising Models for Semantic Manipulation](https://mlsanity.com/paper/2503.06966/) (2025)
- [CyclePose -- Leveraging Cycle-Consistency for Annotation-Free Nuclei Segmentation in Fluorescence Microscopy](https://mlsanity.com/paper/2503.11266/) (2025)
- [Atomic Depth Estimation From Noisy Electron Microscopy Data Via Deep Learning](https://mlsanity.com/paper/2601.17046/) (2026)
- [Vision-Language Controlled Deep Unfolding for Joint Medical Image Restoration and Segmentation](https://mlsanity.com/paper/2601.23103/) (2026)
- [High-Resolution Image Reconstruction with Unsupervised Learning and Noisy Data Applied to Ion-Beam Dynamics for Particle Accelerators](https://mlsanity.com/paper/2603.06689/) (2026)
- [ImPartial: Multi-channel Whole-Cell Segmentation using Partial Annotations](https://mlsanity.com/paper/2605.24128/) (2026)
- [Joint Medical Image Enhancement and Segmentation with Diffusion-based Symbiotic Information Interaction](https://mlsanity.com/paper/2607.00058/) (2026)

## Abstract (arXiv)

Microscopy image analysis often requires the segmentation of objects, but training data for this task is typically scarce and hard to obtain. Here we propose DenoiSeg, a new method that can be trained end-to-end on only a few annotated ground truth segmentations. We achieve this by extending Noise2Void, a self-supervised denoising scheme that can be trained on noisy images alone, to also predict dense 3-class segmentations. The reason for the success of our method is that segmentation can profit from denoising, especially when performed jointly within the same network. The network becomes a denoising expert by seeing all available raw data, while co-learning to segment, even if only a few segmentation labels are available. This hypothesis is additionally fueled by our observation that the best segmentation results on high quality (very low noise) raw data are obtained when moderate amounts of synthetic noise are added. This renders the denoising-task non-trivial and unleashes the desired co-learning effect. We believe that DenoiSeg offers a viable way to circumvent the tremendous hunger for high quality training data and effectively enables few-shot learning of dense segmentations.