# Integrating Anatomical Priors into a Causal Diffusion Model

arXiv: https://arxiv.org/abs/2509.09054 · HTML: https://mlsanity.com/paper/2509.09054/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 44 of its 88 references that are in the map.
- First posted on arXiv 10 Sep 2025.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 30 of 44)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [Diffusion Autoencoders: Toward a Meaningful and Decodable Representation](https://mlsanity.com/paper/2111.15640/) (2021)
- [Score-based diffusion models for accelerated MRI](https://mlsanity.com/paper/2110.05243/) (2021)
- [Brain Imaging Generation with Latent Diffusion Models](https://mlsanity.com/paper/2209.07162/) (2022)
- [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022)
- [Deep Structural Causal Models for Tractable Counterfactual Inference](https://mlsanity.com/paper/2006.06485/) (2020)
- [Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks](https://mlsanity.com/paper/1807.10225/) (2018)
- [Image Synthesis in Multi-Contrast MRI with Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1802.01221/) (2018)
- [Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Network](https://mlsanity.com/paper/1904.12200/) (2019)
- [Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Networks](https://mlsanity.com/paper/1908.02498/) (2019)
- [Hierarchical Amortized Training for Memory-efficient High Resolution 3D GAN](https://mlsanity.com/paper/2008.01910/) (2020)
- [SADM: Sequence-Aware Diffusion Model for Longitudinal Medical Image Generation](https://mlsanity.com/paper/2212.08228/) (2022)
- [High Fidelity Image Counterfactuals with Probabilistic Causal Models](https://mlsanity.com/paper/2306.15764/) (2023)
- [Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain MRI datasets](https://mlsanity.com/paper/2209.02032/) (2022)
- [MedSyn: Text-guided Anatomy-aware Synthesis of High-Fidelity 3D CT Images](https://mlsanity.com/paper/2310.03559/) (2023)
- [Equitable modelling of brain imaging by counterfactual augmentation with morphologically constrained 3D deep generative models](https://mlsanity.com/paper/2111.14923/) (2021)
- [Degenerative Adversarial NeuroImage Nets for Brain Scan Simulations: Application in Ageing and Dementia](https://mlsanity.com/paper/1912.01526/) (2019)
- [MedGen3D: A Deep Generative Framework for Paired 3D Image and Mask Generation](https://mlsanity.com/paper/2304.04106/) (2023)

## Built on by (most cited first, 1 of 1)

- [Modality-Aware and Anatomical Vector-Quantized Autoencoding for Multimodal Brain MRI](https://mlsanity.com/paper/2604.05171/) (2026)

## Abstract (arXiv)

3D brain MRI studies often examine subtle morphometric differences between cohorts that are hard to detect visually. Given the high cost of MRI acquisition, these studies could greatly benefit from image syntheses, particularly counterfactual image generation, as seen in other domains, such as computer vision. However, counterfactual models struggle to produce anatomically plausible MRIs due to the lack of explicit inductive biases to preserve fine-grained anatomical details. This shortcoming arises from the training of the models aiming to optimize for the overall appearance of the images (e.g., via cross-entropy) rather than preserving subtle, yet medically relevant, local variations across subjects. To preserve subtle variations, we propose to explicitly integrate anatomical constraints on a voxel-level as prior into a generative diffusion framework. Called Probabilistic Causal Graph Model (PCGM), the approach captures anatomical constraints via a probabilistic graph module and translates those constraints into spatial binary masks of regions where subtle variations occur. The masks (encoded by a 3D extension of ControlNet) constrain a novel counterfactual denoising UNet, whose encodings are then transferred into high-quality brain MRIs via our 3D diffusion decoder. Extensive experiments on multiple datasets demonstrate that PCGM generates structural brain MRIs of higher quality than several baseline approaches. Furthermore, we show for the first time that brain measurements extracted from counterfactuals (generated by PCGM) replicate the subtle effects of a disease on cortical brain regions previously reported in the neuroscience literature. This achievement is an important milestone in the use of synthetic MRIs in studies investigating subtle morphological differences.