# 3D-StyleGAN: A Style-Based Generative Adversarial Network for Generative Modeling of Three-Dimensional Medical Images

arXiv: https://arxiv.org/abs/2107.09700 · HTML: https://mlsanity.com/paper/2107.09700/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 20 Jul 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [Conditional Image Synthesis With Auxiliary Classifier GANs](https://mlsanity.com/paper/1610.09585/) (2016)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Networks](https://mlsanity.com/paper/1908.02498/) (2019)
- [Anatomical Priors in Convolutional Networks for Unsupervised Biomedical Segmentation](https://mlsanity.com/paper/1903.03148/) (2019)
- [Modelling the Distribution of 3D Brain MRI using a 2D Slice VAE](https://mlsanity.com/paper/2007.04780/) (2020)

## Built on by (most cited first, 30 of 32)

- [Conditional Diffusion Models for Semantic 3D Brain MRI Synthesis](https://mlsanity.com/paper/2305.18453/) (2023)
- [WDM: 3D Wavelet Diffusion Models for High-Resolution Medical Image Synthesis](https://mlsanity.com/paper/2402.19043/) (2024)
- [Equitable modelling of brain imaging by counterfactual augmentation with morphologically constrained 3D deep generative models](https://mlsanity.com/paper/2111.14923/) (2021)
- [Deep Generative Models for 3D Medical Image Synthesis](https://mlsanity.com/paper/2410.17664/) (2024)
- [Metadata-Conditioned Generative Models to Synthesize Anatomically-Plausible 3D Brain MRIs](https://mlsanity.com/paper/2310.04630/) (2023)
- [Assessing the ability of generative adversarial networks to learn canonical medical image statistics](https://mlsanity.com/paper/2204.12007/) (2022)
- [Three-dimensional microstructure generation using generative adversarial neural networks in the context of continuum micromechanics](https://mlsanity.com/paper/2206.01693/) (2022)
- [Inflating 2D Convolution Weights for Efficient Generation of 3D Medical Images](https://mlsanity.com/paper/2208.03934/) (2022)
- [Multiscale Metamorphic VAE for 3D Brain MRI Synthesis](https://mlsanity.com/paper/2301.03588/) (2023)
- [Generating 3D Brain Tumor Regions in MRI using Vector-Quantization Generative Adversarial Networks](https://mlsanity.com/paper/2310.01251/) (2023)
- [3D Brain and Heart Volume Generative Models: A Survey](https://mlsanity.com/paper/2210.05952/) (2022)
- [A Self-attention Guided Multi-scale Gradient GAN for Diversified X-ray Image Synthesis](https://mlsanity.com/paper/2210.06334/) (2022)
- [Mining the manifolds of deep generative models for multiple data-consistent solutions of ill-posed tomographic imaging problems](https://mlsanity.com/paper/2202.05311/) (2022)
- [Explicit Temporal Embedding in Deep Generative Latent Models for Longitudinal Medical Image Synthesis](https://mlsanity.com/paper/2301.05465/) (2023)
- [Metrics to Quantify Global Consistency in Synthetic Medical Images](https://mlsanity.com/paper/2308.00402/) (2023)
- [Improved α-GAN architecture for generating 3D connected volumes with an application to radiosurgery treatment planning](https://mlsanity.com/paper/2207.11223/) (2022)
- [Merging synthetic and real embryo data for advanced AI predictions](https://mlsanity.com/paper/2412.01255/) (2024)
- [Semantics versus Identity: A Divide-and-Conquer Approach towards Adjustable Medical Image De-Identification](https://mlsanity.com/paper/2507.21703/) (2025)
- [Pixel-Level Residual Diffusion Transformer: Scalable 3D CT Volume Generation](https://mlsanity.com/paper/2606.20112/) (2026)
- [Evaluating Procedures for Establishing Generative Adversarial Network-based Stochastic Image Models in Medical Imaging](https://mlsanity.com/paper/2204.03547/) (2022)
- [USB: Unified Synthetic Brain Framework for Bidirectional Pathology-Healthy Generation and Editing](https://mlsanity.com/paper/2512.00269/) (2025)
- [Highly Accurate FMRI ADHD Classification using time distributed multi modal 3D CNNs](https://mlsanity.com/paper/2205.11993/) (2022)
- [Optimizing CAD Models with Latent Space Manipulation](https://mlsanity.com/paper/2303.12739/) (2023)
- [Causal Image Synthesis of Brain MR in 3D](https://mlsanity.com/paper/2303.14349/) (2023)
- [Three-dimensional Bone Image Synthesis with Generative Adversarial Networks](https://mlsanity.com/paper/2310.17216/) (2023)
- [Evaluating Utility of Memory Efficient Medical Image Generation: A Study on Lung Nodule Segmentation](https://mlsanity.com/paper/2410.12542/) (2024)
- [Longitudinal Causal Image Synthesis](https://mlsanity.com/paper/2410.17691/) (2024)
- [Comparative clinical evaluation of "memory-efficient" synthetic 3d generative adversarial networks (gan) head-to-head to state of art: results on computed tomography of the chest](https://mlsanity.com/paper/2501.15572/) (2025)
- [Memory-Efficient 3D High-Resolution Medical Image Synthesis Using CRF-Guided GANs](https://mlsanity.com/paper/2503.10899/) (2025)
- [TumorGen: Boundary-Aware Tumor-Mask Synthesis with Rectified Flow Matching](https://mlsanity.com/paper/2505.24687/) (2025)

## Abstract (arXiv)

Image synthesis via Generative Adversarial Networks (GANs) of three-dimensional (3D) medical images has great potential that can be extended to many medical applications, such as, image enhancement and disease progression modeling. However, current GAN technologies for 3D medical image synthesis need to be significantly improved to be readily adapted to real-world medical problems. In this paper, we extend the state-of-the-art StyleGAN2 model, which natively works with two-dimensional images, to enable 3D image synthesis. In addition to the image synthesis, we investigate the controllability and interpretability of the 3D-StyleGAN via style vectors inherited form the original StyleGAN2 that are highly suitable for medical applications: (i) the latent space projection and reconstruction of unseen real images, and (ii) style mixing. We demonstrate the 3D-StyleGAN's performance and feasibility with ~12,000 three-dimensional full brain MR T1 images, although it can be applied to any 3D volumetric images. Furthermore, we explore different configurations of hyperparameters to investigate potential improvement of the image synthesis with larger networks. The codes and pre-trained networks are available online: https://github.com/sh4174/3DStyleGAN.