# Cascaded Diffusion Models for 2D and 3D Microscopy Image Synthesis to Enhance Cell Segmentation

arXiv: https://arxiv.org/abs/2411.11515 · HTML: https://mlsanity.com/paper/2411.11515/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 18 Nov 2024.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 12 of 12)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [SyncDreamer: Generating Multiview-consistent Images from a Single-view Image](https://mlsanity.com/paper/2309.03453/) (2023)
- [Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models](https://mlsanity.com/paper/2305.10474/) (2023)
- [Three Dimensional Fluorescence Microscopy Image Synthesis and Segmentation](https://mlsanity.com/paper/1801.07198/) (2018)
- [Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Sets](https://mlsanity.com/paper/2301.10227/) (2023)
- [3D fluorescence microscopy data synthesis for segmentation and benchmarking](https://mlsanity.com/paper/2107.10180/) (2021)
- [Annotated Biomedical Video Generation using Denoising Diffusion Probabilistic Models and Flow Fields](https://mlsanity.com/paper/2403.17808/) (2024)

## Built on by (most cited first, 2 of 2)

- [A Pragmatic Note on Evaluating Generative Models with Fr'echet Inception Distance for Retinal Image Synthesis](https://mlsanity.com/paper/2502.17160/) (2025)
- [CellStyle: Improved Zero-Shot Cell Segmentation via Style Transfer](https://mlsanity.com/paper/2503.08603/) (2025)

## Abstract (arXiv)

Automated cell segmentation in microscopy images is essential for biomedical research, yet conventional methods are labor-intensive and prone to error. While deep learning-based approaches have proven effective, they often require large annotated datasets, which are scarce due to the challenges of manual annotation. To overcome this, we propose a novel framework for synthesizing densely annotated 2D and 3D cell microscopy images using cascaded diffusion models. Our method synthesizes 2D and 3D cell masks from sparse 2D annotations using multi-level diffusion models and NeuS, a 3D surface reconstruction approach. Following that, a pretrained 2D Stable Diffusion model is finetuned to generate realistic cell textures and the final outputs are combined to form cell populations. We show that training a segmentation model with a combination of our synthetic data and real data improves cell segmentation performance by up to 9% across multiple datasets. Additionally, the FID scores indicate that the synthetic data closely resembles real data. The code for our proposed approach will be available at https://github.com/ruveydayilmaz0/cascadeddiffusion.