# 3D fluorescence microscopy data synthesis for segmentation and benchmarking

arXiv: https://arxiv.org/abs/2107.10180 · HTML: https://mlsanity.com/paper/2107.10180/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 21 Jul 2021.

## Builds on (most cited first, 12 of 12)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [CNN-based Preprocessing to Optimize Watershed-based Cell Segmentation in 3D Confocal Microscopy Images](https://mlsanity.com/paper/1810.06933/) (2018)
- [CellCycleGAN: Spatiotemporal Microscopy Image Synthesis of Cell Populations using Statistical Shape Models and Conditional GANs](https://mlsanity.com/paper/2010.12011/) (2020)
- [Robust 3D Cell Segmentation: Extending the View of Cellpose](https://mlsanity.com/paper/2105.00794/) (2021)
- [Spherical Harmonics for Shape-Constrained 3D Cell Segmentation](https://mlsanity.com/paper/2010.12369/) (2020)
- [SEGMENT3D: A Web-based Application for Collaborative Segmentation of 3D images used in the Shoot Apical Meristem](https://mlsanity.com/paper/1710.09933/) (2017)
- [Semi-Automatic Generation of Tight Binary Masks and Non-Convex Isosurfaces for Quantitative Analysis of 3D Biological Samples](https://mlsanity.com/paper/2001.11469/) (2020)

## Built on by (most cited first, 8 of 8)

- [Medical Diffusion: Denoising Diffusion Probabilistic Models for 3D Medical Image Generation](https://mlsanity.com/paper/2211.03364/) (2022)
- [AnyStar: Domain randomized universal star-convex 3D instance segmentation](https://mlsanity.com/paper/2307.07044/) (2023)
- [Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Sets](https://mlsanity.com/paper/2301.10227/) (2023)
- [Improving 3D deep learning segmentation with biophysically motivated cell synthesis](https://mlsanity.com/paper/2408.16471/) (2024)
- [Cascaded Diffusion Models for 2D and 3D Microscopy Image Synthesis to Enhance Cell Segmentation](https://mlsanity.com/paper/2411.11515/) (2024)
- [Bayesian Optimization for Design Parameters of 3D Image Data Analysis](https://mlsanity.com/paper/2602.15660/) (2026)
- [Cell Phantom Video Generation in Elliptical Fourier Descriptor Domain](https://mlsanity.com/paper/2605.22563/) (2026)
- [Stitching and dimensionality effects on large artificially generated volume datasets](https://mlsanity.com/paper/2606.20095/) (2026)

## Abstract (arXiv)

Automated image processing approaches are indispensable for many biomedical experiments and help to cope with the increasing amount of microscopy image data in a fast and reproducible way. Especially state-of-the-art deep learning-based approaches most often require large amounts of annotated training data to produce accurate and generalist outputs, but they are often compromised by the general lack of those annotated data sets. In this work, we propose how conditional generative adversarial networks can be utilized to generate realistic image data for 3D fluorescence microscopy from annotation masks of 3D cellular structures. In combination with mask simulation approaches, we demonstrate the generation of fully-annotated 3D microscopy data sets that we make publicly available for training or benchmarking. An additional positional conditioning of the cellular structures enables the reconstruction of position-dependent intensity characteristics and allows to generate image data of different quality levels. A patch-wise working principle and a subsequent full-size reassemble strategy is used to generate image data of arbitrary size and different organisms. We present this as a proof-of-concept for the automated generation of fully-annotated training data sets requiring only a minimum of manual interaction to alleviate the need of manual annotations.