# Diffusion-based Data Augmentation for Nuclei Image Segmentation

arXiv: https://arxiv.org/abs/2310.14197 · HTML: https://mlsanity.com/paper/2310.14197/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 21 of its 30 references that are in the map.
- First posted on arXiv 22 Oct 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images](https://mlsanity.com/paper/1812.06499/) (2018)
- [SinGAN: Learning a Generative Model from a Single Natural Image](https://mlsanity.com/paper/1905.01164/) (2019)
- [SegDiff: Image Segmentation with Diffusion Probabilistic Models](https://mlsanity.com/paper/2112.00390/) (2021)
- [Motion Guided Attention for Video Salient Object Detection](https://mlsanity.com/paper/1909.07061/) (2019)
- [Unsupervised Instance Segmentation in Microscopy Images via Panoptic Domain Adaptation and Task Re-weighting](https://mlsanity.com/paper/2005.02066/) (2020)
- [Cubic LSTMs for Video Prediction](https://mlsanity.com/paper/1904.09412/) (2019)
- [Which Pixel to Annotate: a Label-Efficient Nuclei Segmentation Framework](https://mlsanity.com/paper/2212.10305/) (2022)
- [Depthwise Non-local Module for Fast Salient Object Detection Using a Single Thread](https://mlsanity.com/paper/2001.08057/) (2020)
- [Robust Real-World Image Super-Resolution against Adversarial Attacks](https://mlsanity.com/paper/2208.00428/) (2022)
- [Context-Aware Semantic Inpainting](https://mlsanity.com/paper/1712.07778/) (2017)

## Built on by (most cited first, 30 of 31)

- [URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Images](https://mlsanity.com/paper/2405.11656/) (2024)
- [Image Segmentation in Foundation Model Era: A Survey](https://mlsanity.com/paper/2408.12957/) (2024)
- [A Survey on Data Augmentation in Large Model Era](https://mlsanity.com/paper/2401.15422/) (2024)
- [FlowSDF: Flow Matching for Medical Image Segmentation Using Distance Transforms](https://mlsanity.com/paper/2405.18087/) (2024)
- [Relaxed Equivariance via Multitask Learning](https://mlsanity.com/paper/2410.17878/) (2024)
- [Cell Graph Transformer for Nuclei Classification](https://mlsanity.com/paper/2402.12946/) (2024)
- [PathDiff: Histopathology Image Synthesis with Unpaired Text and Mask Conditions](https://mlsanity.com/paper/2506.23440/) (2025)
- [Controllable and Efficient Multi-Class Pathology Nuclei Data Augmentation using Text-Conditioned Diffusion Models](https://mlsanity.com/paper/2407.14426/) (2024)
- [Co-synthesis of Histopathology Nuclei Image-Label Pairs using a Context-Conditioned Joint Diffusion Model](https://mlsanity.com/paper/2407.14434/) (2024)
- [TRACE: Temporally Reliable Anatomically-Conditioned 3D CT Generation with Enhanced Efficiency](https://mlsanity.com/paper/2507.00802/) (2025)
- [Diffuse-UDA: Addressing Unsupervised Domain Adaptation in Medical Image Segmentation with Appearance and Structure Aligned Diffusion Models](https://mlsanity.com/paper/2408.05985/) (2024)
- [Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challenges](https://mlsanity.com/paper/2505.10993/) (2025)
- [Learning Inter-Atomic Potentials without Explicit Equivariance](https://mlsanity.com/paper/2510.00027/) (2025)
- [A Comprehensive Survey on Diffusion Models and Their Applications](https://mlsanity.com/paper/2408.10207/) (2024)
- [Synthetic Augmentation for Anatomical Landmark Localization using DDPMs](https://mlsanity.com/paper/2410.12489/) (2024)
- [Label-Efficient Data Augmentation with Video Diffusion Models for Guidewire Segmentation in Cardiac Fluoroscopy](https://mlsanity.com/paper/2412.16050/) (2024)
- [UniCell: Universal Cell Nucleus Classification via Prompt Learning](https://mlsanity.com/paper/2402.12938/) (2024)
- [Unleashing the Potential of Synthetic Images: A Study on Histopathology Image Classification](https://mlsanity.com/paper/2409.16002/) (2024)
- [AdaptDiff: Cross-Modality Domain Adaptation via Weak Conditional Semantic Diffusion for Retinal Vessel Segmentation](https://mlsanity.com/paper/2410.04648/) (2024)
- [Diffusion-based Data Augmentation and Knowledge Distillation with Generated Soft Labels Solving Data Scarcity Problems of SAR Oil Spill Segmentation](https://mlsanity.com/paper/2412.08116/) (2024)
- [Explainable, Multi-modal Wound Infection Classification from Images Augmented with Generated Captions](https://mlsanity.com/paper/2502.20277/) (2025)
- [Inpainting is All You Need: A Diffusion-based Augmentation Method for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2506.23038/) (2025)
- [LoGex: Improved tail detection of extremely rare histopathology classes via guided diffusion](https://mlsanity.com/paper/2409.01317/) (2024)
- [HistoSmith: Single-Stage Histology Image-Label Generation via Conditional Latent Diffusion for Enhanced Cell Segmentation and Classification](https://mlsanity.com/paper/2502.08754/) (2025)
- [Instance Migration Diffusion for Nuclear Instance Segmentation in Pathology](https://mlsanity.com/paper/2504.01577/) (2025)
- [MSDM: Generating Task-Specific Pathology Images with a Multimodal Conditioned Diffusion Model for Cell and Nuclei Segmentation](https://mlsanity.com/paper/2510.09121/) (2025)
- [Weak-to-Strong Generalization Enables Fully Automated De Novo Training of Multi-head Mask-RCNN Model for Segmenting Densely Overlapping Cell Nuclei in Multiplex Whole-slice Brain Images](https://mlsanity.com/paper/2512.11722/) (2025)
- [MAUGen: A Unified Diffusion Approach for Multi-Identity Facial Expression and AU Label Generation](https://mlsanity.com/paper/2602.00583/) (2026)
- [Controllable Histopathology Image Synthesis with Training-free Structural Initialization and Textural Modulation](https://mlsanity.com/paper/2606.27935/) (2026)
- [Assessment of Conditional Diffusion Model for Synthetic Histopathology Image Generation](https://mlsanity.com/paper/2608.03990/) (2026)

## Abstract (arXiv)

Nuclei segmentation is a fundamental but challenging task in the quantitative analysis of histopathology images. Although fully-supervised deep learning-based methods have made significant progress, a large number of labeled images are required to achieve great segmentation performance. Considering that manually labeling all nuclei instances for a dataset is inefficient, obtaining a large-scale human-annotated dataset is time-consuming and labor-intensive. Therefore, augmenting a dataset with only a few labeled images to improve the segmentation performance is of significant research and application value. In this paper, we introduce the first diffusion-based augmentation method for nuclei segmentation. The idea is to synthesize a large number of labeled images to facilitate training the segmentation model. To achieve this, we propose a two-step strategy. In the first step, we train an unconditional diffusion model to synthesize the Nuclei Structure that is defined as the representation of pixel-level semantic and distance transform. Each synthetic nuclei structure will serve as a constraint on histopathology image synthesis and is further post-processed to be an instance map. In the second step, we train a conditioned diffusion model to synthesize histopathology images based on nuclei structures. The synthetic histopathology images paired with synthetic instance maps will be added to the real dataset for training the segmentation model. The experimental results show that by augmenting 10% labeled real dataset with synthetic samples, one can achieve comparable segmentation results with the fully-supervised baseline. The code is released in: https://github.com/lhaof/Nudiff