# Metadata-Conditioned Generative Models to Synthesize Anatomically-Plausible 3D Brain MRIs

arXiv: https://arxiv.org/abs/2310.04630 · HTML: https://mlsanity.com/paper/2310.04630/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 53 references that are in the map.
- First posted on arXiv 7 Oct 2023.

## Builds on (most cited first, 25 of 25)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [Brain Imaging Generation with Latent Diffusion Models](https://mlsanity.com/paper/2209.07162/) (2022)
- [Deep Structural Causal Models for Tractable Counterfactual Inference](https://mlsanity.com/paper/2006.06485/) (2020)
- [Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks](https://mlsanity.com/paper/1807.10225/) (2018)
- [Image Synthesis in Multi-Contrast MRI with Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1802.01221/) (2018)
- [Predicting brain age with deep learning from raw imaging data results in a reliable and heritable biomarker](https://mlsanity.com/paper/1612.02572/) (2016)
- [Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codes](https://mlsanity.com/paper/2111.12701/) (2021)
- [Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Networks](https://mlsanity.com/paper/1908.02498/) (2019)
- [Learning Conditional Deformable Templates with Convolutional Networks](https://mlsanity.com/paper/1908.02738/) (2019)
- [Hierarchical Amortized Training for Memory-efficient High Resolution 3D GAN](https://mlsanity.com/paper/2008.01910/) (2020)
- [3D-StyleGAN: A Style-Based Generative Adversarial Network for Generative Modeling of Three-Dimensional Medical Images](https://mlsanity.com/paper/2107.09700/) (2021)
- [Longitudinal Self-Supervised Learning](https://mlsanity.com/paper/2006.06930/) (2020)
- [Generating Realistic Brain MRIs via a Conditional Diffusion Probabilistic Model](https://mlsanity.com/paper/2212.08034/) (2022)
- [Metadata Normalization](https://mlsanity.com/paper/2104.09052/) (2021)
- [Confounder-Aware Visualization of ConvNets](https://mlsanity.com/paper/1907.12727/) (2019)

## Built on by (most cited first, 12 of 12)

- [LeFusion: Controllable Pathology Synthesis via Lesion-Focused Diffusion Models](https://mlsanity.com/paper/2403.14066/) (2024)
- [Box It to Bind It: Unified Layout Control and Attribute Binding in T2I Diffusion Models](https://mlsanity.com/paper/2402.17910/) (2024)
- [Latent Causal Modeling for 3D Brain MRI Counterfactuals](https://mlsanity.com/paper/2409.05585/) (2024)
- [WASABI: A Metric for Evaluating Morphometric Plausibility of Synthetic Brain MRIs](https://mlsanity.com/paper/2504.21771/) (2025)
- [Evaluation of 3D Counterfactual Brain MRI Generation](https://mlsanity.com/paper/2508.02880/) (2025)
- [Anatomically Guided Latent Diffusion for Brain MRI Progression Modeling](https://mlsanity.com/paper/2601.14584/) (2026)
- [Integrating Anatomical Priors into a Causal Diffusion Model](https://mlsanity.com/paper/2509.09054/) (2025)
- [Cycle Diffusion Model for Counterfactual Image Generation](https://mlsanity.com/paper/2509.24267/) (2025)
- [Cortex-Grounded Diffusion Models for Brain Image Generation](https://mlsanity.com/paper/2601.19498/) (2026)
- [Beyond Representation Learning: A Systematic Study of Joint-Embedding Predictive Generation for 3D Brain MRI](https://mlsanity.com/paper/2608.28787/) (2026)
- [Generating Novel Brain Morphology by Deforming Learned Templates](https://mlsanity.com/paper/2503.03778/) (2025)
- [Modality-Aware and Anatomical Vector-Quantized Autoencoding for Multimodal Brain MRI](https://mlsanity.com/paper/2604.05171/) (2026)

## Abstract (arXiv)

Generative AI models hold great potential in creating synthetic brain MRIs that advance neuroimaging studies by, for example, enriching data diversity. However, the mainstay of AI research only focuses on optimizing the visual quality (such as signal-to-noise ratio) of the synthetic MRIs while lacking insights into their relevance to neuroscience. To gain these insights with respect to T1-weighted MRIs, we first propose a new generative model, BrainSynth, to synthesize metadata-conditioned (e.g., age- and sex-specific) MRIs that achieve state-of-the-art visual quality. We then extend our evaluation with a novel procedure to quantify anatomical plausibility, i.e., how well the synthetic MRIs capture macrostructural properties of brain regions, and how accurately they encode the effects of age and sex. Results indicate that more than half of the brain regions in our synthetic MRIs are anatomically accurate, i.e., with a small effect size between real and synthetic MRIs. Moreover, the anatomical plausibility varies across cortical regions according to their geometric complexity. As is, our synthetic MRIs can significantly improve the training of a Convolutional Neural Network to identify accelerated aging effects in an independent study. These results highlight the opportunities of using generative AI to aid neuroimaging research and point to areas for further improvement.