# Longitudinal Causal Image Synthesis

arXiv: https://arxiv.org/abs/2410.17691 · HTML: https://mlsanity.com/paper/2410.17691/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 116 references that are in the map.
- First posted on arXiv 23 Oct 2024.

## Builds on (most cited first, 30 of 31)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [Brain Imaging Generation with Latent Diffusion Models](https://mlsanity.com/paper/2209.07162/) (2022)
- [Deep Structural Causal Models for Tractable Counterfactual Inference](https://mlsanity.com/paper/2006.06485/) (2020)
- [Causal Inference in the Presence of Latent Variables and Selection Bias](https://mlsanity.com/paper/1302.4983/) (2013)
- [Image Synthesis in Multi-Contrast MRI with Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1802.01221/) (2018)
- [Generative AI for Medical Imaging: extending the MONAI Framework](https://mlsanity.com/paper/2307.15208/) (2023)
- [Hierarchical Amortized Training for Memory-efficient High Resolution 3D GAN](https://mlsanity.com/paper/2008.01910/) (2020)
- [What is Healthy? Generative Counterfactual Diffusion for Lesion Localization](https://mlsanity.com/paper/2207.12268/) (2022)
- [Visual Feature Attribution using Wasserstein GANs](https://mlsanity.com/paper/1711.08998/) (2017)
- [Unified Multi-Modal Image Synthesis for Missing Modality Imputation](https://mlsanity.com/paper/2304.05340/) (2023)
- [Bidirectional Mapping Generative Adversarial Networks for Brain MR to PET Synthesis](https://mlsanity.com/paper/2008.03483/) (2020)
- [3D-StyleGAN: A Style-Based Generative Adversarial Network for Generative Modeling of Three-Dimensional Medical Images](https://mlsanity.com/paper/2107.09700/) (2021)
- [A Structural Causal Model for MR Images of Multiple Sclerosis](https://mlsanity.com/paper/2103.03158/) (2021)
- [Diagnose Like a Radiologist: Hybrid Neuro-Probabilistic Reasoning for Attribute-Based Medical Image Diagnosis](https://mlsanity.com/paper/2208.09282/) (2022)
- [Structure-Preserving Synthesis: MaskGAN for Unpaired MR-CT Translation](https://mlsanity.com/paper/2307.16143/) (2023)
- [Learn-Explain-Reinforce: Counterfactual Reasoning and Its Guidance to Reinforce an Alzheimer's Disease Diagnosis Model](https://mlsanity.com/paper/2108.09451/) (2021)
- [Counterfactual Image Synthesis for Discovery of Personalized Predictive Image Markers](https://mlsanity.com/paper/2208.02311/) (2022)
- [Towards Causality-Aware Inferring: A Sequential Discriminative Approach for Medical Diagnosis](https://mlsanity.com/paper/2003.06534/) (2020)
- [Personalized Gaussian Processes for Forecasting of Alzheimer's Disease Assessment Scale-Cognition Sub-Scale (ADAS-Cog13)](https://mlsanity.com/paper/1802.08561/) (2018)
- [Multi-task longitudinal forecasting with missing values on Alzheimer's Disease](https://mlsanity.com/paper/2201.05040/) (2022)

## Abstract (arXiv)

Clinical decision-making relies heavily on causal reasoning and longitudinal analysis. For example, for a patient with Alzheimer's disease (AD), how will the brain grey matter atrophy in a year if intervened on the A-beta level in cerebrospinal fluid? The answer is fundamental to diagnosis and follow-up treatment. However, this kind of inquiry involves counterfactual medical images which can not be acquired by instrumental or correlation-based image synthesis models. Yet, such queries require counterfactual medical images, not obtainable through standard image synthesis models. Hence, a causal longitudinal image synthesis (CLIS) method, enabling the synthesis of such images, is highly valuable. However, building a CLIS model confronts three primary yet unmet challenges: mismatched dimensionality between high-dimensional images and low-dimensional tabular variables, inconsistent collection intervals of follow-up data, and inadequate causal modeling capability of existing causal graph methods for image data. In this paper, we established a tabular-visual causal graph (TVCG) for CLIS overcoming these challenges through a novel integration of generative imaging, continuous-time modeling, and structural causal models combined with a neural network. We train our CLIS based on the ADNI dataset and evaluate it on two other AD datasets, which illustrate the outstanding yet controllable quality of the synthesized images and the contributions of synthesized MRI to the characterization of AD progression, substantiating the reliability and utility in clinics.