# Abdominal Ultrasound Simulation from Semantic Labels using Paired Label-to-Physics-Based Image Translation

arXiv: https://arxiv.org/abs/2610.08849 · HTML: https://mlsanity.com/paper/2610.08849/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 50 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Builds on (most cited first, 21 of 21)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [Semantic Image Synthesis via Diffusion Models](https://mlsanity.com/paper/2207.00050/) (2022)
- [GANs for Medical Image Synthesis: An Empirical Study](https://mlsanity.com/paper/2105.05318/) (2021)
- [Sketch guided and progressive growing GAN for realistic and editable ultrasound image synthesis](https://mlsanity.com/paper/2204.06929/) (2022)
- [Echo from noise: synthetic ultrasound image generation using diffusion models for real image segmentation](https://mlsanity.com/paper/2305.05424/) (2023)
- [Content-Preserving Unpaired Translation from Simulated to Realistic Ultrasound Images](https://mlsanity.com/paper/2103.05745/) (2021)
- [Stochastic Conditional Diffusion Models for Robust Semantic Image Synthesis](https://mlsanity.com/paper/2402.16506/) (2024)
- [Diffusion as Sound Propagation: Physics-inspired Model for Ultrasound Image Generation](https://mlsanity.com/paper/2407.05428/) (2024)
- [Ultrasound Image Generation using Latent Diffusion Models](https://mlsanity.com/paper/2502.08580/) (2025)
- [Weakly-supervised High-fidelity Ultrasound Video Synthesis with Feature Decoupling](https://mlsanity.com/paper/2207.00474/) (2022)
- [FetalFlex: Anatomy-Guided Diffusion Model for Flexible Control on Fetal Ultrasound Image Synthesis](https://mlsanity.com/paper/2503.14906/) (2025)
- [Deep Image Translation for Enhancing Simulated Ultrasound Images](https://mlsanity.com/paper/2006.10850/) (2020)
- [Learning Ultrasound Rendering from Cross-Sectional Model Slices for Simulated Training](https://mlsanity.com/paper/2101.08339/) (2021)
- [Unpaired Translation from Semantic Label Maps to Images by Leveraging Domain-Specific Simulations](https://mlsanity.com/paper/2302.10698/) (2023)
- [S-CycleGAN: Semantic Segmentation Enhanced CT-Ultrasound Image-to-Image Translation for Robotic Ultrasonography](https://mlsanity.com/paper/2406.01191/) (2024)

## Abstract (arXiv)

Purpose: Current abdominal ultrasound (US) simulation methods often require CT-based anatomical references for ray-casting, limiting deformation and pathology variability. We propose a learning-based pipeline trained to predict physics-based images derived from CT scans from semantic labels, enabling controlled simulations without patient-specific CT volumes at inference time. Methods: We introduce a two-stage pipeline that maps anatomical segmentations to realistic US images through a simplified US image. Stage~I synthesizes this image from semantic labels using models trained on CT-based ray-casting outputs. Stage~II refines it into a realistic US scan using anatomically guided unpaired translation. Deformations and pathologies are generated by editing anatomical maps. Results: We evaluated Pix2Pix and the Semantic Diffusion Model (SDM) in Stage~I, followed by segmentation-guided CycleGAN (SG-CycleGAN) refinement in Stage~II. SDM significantly outperformed Pix2Pix in morphological metrics, including MAE (19.85 vs. 21.65), SSIM (0.28 vs. 0.24), and mIoU (0.43 vs. 0.29), whereas Pix2Pix yielded better perceptual point estimates (LPIPS: 0.17 vs. 0.19; FID: 0.32 vs. 0.37; KID: 0.25 vs. 0.48). Conclusion: Training paired generative models with physics-based supervision enables approximation of CT-derived ray-casting outputs at inference time directly from semantic labels. Although the pipeline does not require patient-specific CT volumes at inference time, CT-derived segmentations and ray-casting simulations remain necessary to train Stage~I. Once trained, the framework enables controllable healthy and pathological simulations through semantic-map modification.