# Causal Image Synthesis of Brain MR in 3D

arXiv: https://arxiv.org/abs/2303.14349 · HTML: https://mlsanity.com/paper/2303.14349/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 33 references that are in the map.
- First posted on arXiv 25 Mar 2023.

## Builds on (most cited first, 20 of 20)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Med3D: Transfer Learning for 3D Medical Image Analysis](https://mlsanity.com/paper/1904.00625/) (2019)
- [Deep Structural Causal Models for Tractable Counterfactual Inference](https://mlsanity.com/paper/2006.06485/) (2020)
- [Causality matters in medical imaging](https://mlsanity.com/paper/1912.08142/) (2019)
- [Hierarchical Amortized Training for Memory-efficient High Resolution 3D GAN](https://mlsanity.com/paper/2008.01910/) (2020)
- [What is Healthy? Generative Counterfactual Diffusion for Lesion Localization](https://mlsanity.com/paper/2207.12268/) (2022)
- [3D Convolutional Encoder-Decoder Network for Low-Dose CT via Transfer Learning from a 2D Trained Network](https://mlsanity.com/paper/1802.05656/) (2018)
- [Using StyleGAN for Visual Interpretability of Deep Learning Models on Medical Images](https://mlsanity.com/paper/2101.07563/) (2021)
- [3D-StyleGAN: A Style-Based Generative Adversarial Network for Generative Modeling of Three-Dimensional Medical Images](https://mlsanity.com/paper/2107.09700/) (2021)
- [Vox2Vox: 3D-GAN for Brain Tumour Segmentation](https://mlsanity.com/paper/2003.13653/) (2020)
- [A Structural Causal Model for MR Images of Multiple Sclerosis](https://mlsanity.com/paper/2103.03158/) (2021)
- [A Review of Causality for Learning Algorithms in Medical Image Analysis](https://mlsanity.com/paper/2206.05498/) (2022)
- [Memory-efficient GAN-based Domain Translation of High Resolution 3D Medical Images](https://mlsanity.com/paper/2010.03396/) (2020)
- [Virtual Thin Slice: 3D Conditional GAN-based Super-resolution for CT Slice Interval](https://mlsanity.com/paper/1908.11506/) (2019)
- [Counterfactual Image Synthesis for Discovery of Personalized Predictive Image Markers](https://mlsanity.com/paper/2208.02311/) (2022)
- [Bivariate Causal Discovery and its Applications to Gene Expression and Imaging Data Analysis](https://mlsanity.com/paper/1805.04164/) (2018)
- [Semantic Unfolding of StyleGAN Latent Space](https://mlsanity.com/paper/2206.14892/) (2022)

## Abstract (arXiv)

Clinical decision making requires counterfactual reasoning based on a factual medical image and thus necessitates causal image synthesis. To this end, we present a novel method for modeling the causality between demographic variables, clinical indices and brain MR images for Alzheimer's Diseases. Specifically, we leverage a structural causal model to depict the causality and a styled generator to synthesize the image. Furthermore, as a crucial step to reduce modeling complexity and make learning tractable, we propose the use of low dimensional latent feature representation of a high-dimensional 3D image, together with exogenous noise, to build causal relationship between the image and non image variables. We experiment the proposed method based on 1586 subjects and 3683 3D images and synthesize counterfactual brain MR images intervened on certain attributes, such as age, brain volume and cognitive test score. Quantitative metrics and qualitative evaluation of counterfactual images demonstrates the superiority of our generated images.