# SimGen: A Diffusion-Based Framework for Simultaneous Surgical Image and Segmentation Mask Generation

arXiv: https://arxiv.org/abs/2501.09008 · HTML: https://mlsanity.com/paper/2501.09008/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- First posted on arXiv 15 Jan 2025.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Built on by (most cited first, 5 of 5)

- [MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidance](https://mlsanity.com/paper/2503.16421/) (2025)
- [MedSegFactory: Text-Guided Generation of Medical Image-Mask Pairs](https://mlsanity.com/paper/2504.06897/) (2025)
- [Towards Suturing World Models: Learning Predictive Models for Robotic Surgical Tasks](https://mlsanity.com/paper/2503.12531/) (2025)
- [CoSimGen: Controllable Diffusion Model for Simultaneous Image and Mask Generation](https://mlsanity.com/paper/2503.19661/) (2025)
- [Leveraging Image Generators to Address Data Scarcity: The Gen4Regen Dataset for Forest Regeneration Mapping](https://mlsanity.com/paper/2605.05627/) (2026)

## Abstract (arXiv)

Acquiring and annotating surgical data is often resource-intensive, ethical constraining, and requiring significant expert involvement. While generative AI models like text-to-image can alleviate data scarcity, incorporating spatial annotations, such as segmentation masks, is crucial for precision-driven surgical applications, simulation, and education. This study introduces both a novel task and method, SimGen, for Simultaneous Image and Mask Generation. SimGen is a diffusion model based on the DDPM framework and Residual U-Net, designed to jointly generate high-fidelity surgical images and their corresponding segmentation masks. The model leverages cross-correlation priors to capture dependencies between continuous image and discrete mask distributions. Additionally, a Canonical Fibonacci Lattice (CFL) is employed to enhance class separability and uniformity in the RGB space of the masks. SimGen delivers high-fidelity images and accurate segmentation masks, outperforming baselines across six public datasets assessed on image and semantic inception distance metrics. Ablation study shows that the CFL improves mask quality and spatial separation. Downstream experiments suggest generated image-mask pairs are usable if regulations limit human data release for research. This work offers a cost-effective solution for generating paired surgical images and complex labels, advancing surgical AI development by reducing the need for expensive manual annotations.