# Annotated Biomedical Video Generation using Denoising Diffusion Probabilistic Models and Flow Fields

arXiv: https://arxiv.org/abs/2403.17808 · HTML: https://mlsanity.com/paper/2403.17808/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 11 of 11)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Diffusion Probabilistic Models beat GANs on Medical Images](https://mlsanity.com/paper/2212.07501/) (2022)
- [Graph Neural Network for Cell Tracking in Microscopy Videos](https://mlsanity.com/paper/2202.04731/) (2022)
- [Microscopy Cell Segmentation via Convolutional LSTM Networks](https://mlsanity.com/paper/1805.11247/) (2018)
- [Virtual staining for mitosis detection in Breast Histopathology](https://mlsanity.com/paper/2003.07801/) (2020)
- [Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Sets](https://mlsanity.com/paper/2301.10227/) (2023)
- [CellCycleGAN: Spatiotemporal Microscopy Image Synthesis of Cell Populations using Statistical Shape Models and Conditional GANs](https://mlsanity.com/paper/2010.12011/) (2020)
- [Retinal OCT Synthesis with Denoising Diffusion Probabilistic Models for Layer Segmentation](https://mlsanity.com/paper/2311.05479/) (2023)

## Built on by (most cited first, 5 of 5)

- [A Pragmatic Note on Evaluating Generative Models with Fr'echet Inception Distance for Retinal Image Synthesis](https://mlsanity.com/paper/2502.17160/) (2025)
- [Artificial Intelligence for Biomedical Video Generation](https://mlsanity.com/paper/2411.07619/) (2024)
- [Cascaded Diffusion Models for 2D and 3D Microscopy Image Synthesis to Enhance Cell Segmentation](https://mlsanity.com/paper/2411.11515/) (2024)
- [CellStyle: Improved Zero-Shot Cell Segmentation via Style Transfer](https://mlsanity.com/paper/2503.08603/) (2025)
- [Cell Phantom Video Generation in Elliptical Fourier Descriptor Domain](https://mlsanity.com/paper/2605.22563/) (2026)

## Abstract (arXiv)

The segmentation and tracking of living cells play a vital role within the biomedical domain, particularly in cancer research, drug development, and developmental biology. These are usually tedious and time-consuming tasks that are traditionally done by biomedical experts. Recently, to automatize these processes, deep learning based segmentation and tracking methods have been proposed. These methods require large-scale datasets and their full potential is constrained by the scarcity of annotated data in the biomedical imaging domain. To address this limitation, we propose Biomedical Video Diffusion Model (BVDM), capable of generating realistic-looking synthetic microscopy videos. Trained only on a single real video, BVDM can generate videos of arbitrary length with pixel-level annotations that can be used for training data-hungry models. It is composed of a denoising diffusion probabilistic model (DDPM) generating high-fidelity synthetic cell microscopy images and a flow prediction model (FPM) predicting the non-rigid transformation between consecutive video frames. During inference, initially, the DDPM imposes realistic cell textures on synthetic cell masks which are generated based on real data statistics. The flow prediction model predicts the flow field between consecutive masks and applies that to the DDPM output from the previous time frame to create the next one while keeping temporal consistency. BVDM outperforms state-of-the-art synthetic live cell microscopy video generation models. Furthermore, we demonstrate that a sufficiently large synthetic dataset enhances the performance of cell segmentation and tracking models compared to using a limited amount of available real data.