# Segmentation-Aware Image Denoising without Knowing True Segmentation

arXiv: https://arxiv.org/abs/1905.08965 · HTML: https://mlsanity.com/paper/1905.08965/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2019.

## Built on by (most cited first, 6 of 6)

- [Semantic-Guided Zero-Shot Learning for Low-Light Image/Video Enhancement](https://mlsanity.com/paper/2110.00970/) (2021)
- [From Denoising Training to Test-Time Adaptation: Enhancing Domain Generalization for Medical Image Segmentation](https://mlsanity.com/paper/2310.20271/) (2023)
- [SAPNet: Segmentation-Aware Progressive Network for Perceptual Contrastive Deraining](https://mlsanity.com/paper/2111.08892/) (2021)
- [Synergy Between Semantic Segmentation and Image Denoising via Alternate Boosting](https://mlsanity.com/paper/2102.12095/) (2021)
- [Generalized Task-Driven Medical Image Quality Enhancement with Gradient Promotion](https://mlsanity.com/paper/2501.01114/) (2025)
- [High-Resolution Image Reconstruction with Unsupervised Learning and Noisy Data Applied to Ion-Beam Dynamics for Particle Accelerators](https://mlsanity.com/paper/2603.06689/) (2026)

## Abstract (arXiv)

Several recent works discussed application-driven image restoration neural networks, which are capable of not only removing noise in images but also preserving their semantic-aware details, making them suitable for various high-level computer vision tasks as the pre-processing step. However, such approaches require extra annotations for their high-level vision tasks, in order to train the joint pipeline using hybrid losses. The availability of those annotations is yet often limited to a few image sets, potentially restricting the general applicability of these methods to denoising more unseen and unannotated images. Motivated by that, we propose a segmentation-aware image denoising model dubbed U-SAID, based on a novel unsupervised approach with a pixel-wise uncertainty loss. U-SAID does not need any ground-truth segmentation map, and thus can be applied to any image dataset. It generates denoised images with comparable or even better quality, and the denoised results show stronger robustness for subsequent semantic segmentation tasks, when compared to either its supervised counterpart or classical "application-agnostic" denoisers. Moreover, we demonstrate the superior generalizability of U-SAID in three-folds, by plugging its "universal" denoiser without fine-tuning: (1) denoising unseen types of images; (2) denoising as pre-processing for segmenting unseen noisy images; and (3) denoising for unseen high-level tasks. Extensive experiments demonstrate the effectiveness, robustness and generalizability of the proposed U-SAID over various popular image sets.