# Pathology Synthesis of 3D-Consistent Cardiac MR Images using 2D VAEs and GANs

arXiv: https://arxiv.org/abs/2209.04223 · HTML: https://mlsanity.com/paper/2209.04223/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 9 Sep 2022.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Generative Adversarial Network in Medical Imaging: A Review](https://mlsanity.com/paper/1809.07294/) (2018)
- [MedGAN: Medical Image Translation using GANs](https://mlsanity.com/paper/1806.06397/) (2018)
- [Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks](https://mlsanity.com/paper/1807.10225/) (2018)
- [Image Synthesis in Multi-Contrast MRI with Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1802.01221/) (2018)
- [GANs for Medical Image Analysis](https://mlsanity.com/paper/1809.06222/) (2018)
- [Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Networks](https://mlsanity.com/paper/1908.02498/) (2019)
- [Studying Robustness of Semantic Segmentation under Domain Shift in cardiac MRI](https://mlsanity.com/paper/2011.07592/) (2020)
- [Modelling the Distribution of 3D Brain MRI using a 2D Slice VAE](https://mlsanity.com/paper/2007.04780/) (2020)
- [XCAT-GAN for Synthesizing 3D Consistent Labeled Cardiac MR Images on Anatomically Variable XCAT Phantoms](https://mlsanity.com/paper/2007.13408/) (2020)
- [On the effectiveness of GAN generated cardiac MRIs for segmentation](https://mlsanity.com/paper/2005.09026/) (2020)
- [Optimized Automated Cardiac MR Scar Quantification with GAN-Based Data Augmentation](https://mlsanity.com/paper/2109.12940/) (2021)

## Built on by (most cited first, 7 of 7)

- [Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review](https://mlsanity.com/paper/2307.13125/) (2023)
- [Generative AI for Synthetic Data Across Multiple Medical Modalities: A Systematic Review of Recent Developments and Challenges](https://mlsanity.com/paper/2407.00116/) (2024)
- [SDF4CHD: Generative Modeling of Cardiac Anatomies with Congenital Heart Defects](https://mlsanity.com/paper/2311.00332/) (2023)
- [A Unified Framework for Generative Data Augmentation: A Comprehensive Survey](https://mlsanity.com/paper/2310.00277/) (2023)
- [CoNeS: Conditional neural fields with shift modulation for multi-sequence MRI translation](https://mlsanity.com/paper/2309.03320/) (2023)
- [Synthetic Cardiac MRI Image Generation using Deep Generative Models](https://mlsanity.com/paper/2603.24764/) (2026)
- [SV-Cine: Diagnosis-Conditioned Segmentation of Single Ventricle Physiology via Generative Data Augmentation](https://mlsanity.com/paper/2609.12997/) (2026)

## Abstract (arXiv)

We propose a method for synthesizing cardiac magnetic resonance (MR) images with plausible heart pathologies and realistic appearances for the purpose of generating labeled data for the application of supervised deep-learning (DL) training. The image synthesis consists of label deformation and label-to-image translation tasks. The former is achieved via latent space interpolation in a VAE model, while the latter is accomplished via a label-conditional GAN model. We devise three approaches for label manipulation in the latent space of the trained VAE model; i) intra-subject synthesis aiming to interpolate the intermediate slices of a subject to increase the through-plane resolution, ii) inter-subject synthesis aiming to interpolate the geometry and appearance of intermediate images between two dissimilar subjects acquired with different scanner vendors, and iii) pathology synthesis aiming to synthesize a series of pseudo-pathological synthetic subjects with characteristics of a desired heart disease. Furthermore, we propose to model the relationship between 2D slices in the latent space of the VAE prior to reconstruction for generating 3D-consistent subjects from stacking up 2D slice-by-slice generations. We demonstrate that such an approach could provide a solution to diversify and enrich an available database of cardiac MR images and to pave the way for the development of generalizable DL-based image analysis algorithms. We quantitatively evaluate the quality of the synthesized data in an augmentation scenario to achieve generalization and robustness to multi-vendor and multi-disease data for image segmentation. Our code is available at https://github.com/sinaamirrajab/CardiacPathologySynthesis.