# Morphology-preserving Autoregressive 3D Generative Modelling of the Brain

arXiv: https://arxiv.org/abs/2209.03177 · HTML: https://mlsanity.com/paper/2209.03177/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 7 Sep 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Long Range Arena: A Benchmark for Efficient Transformers](https://mlsanity.com/paper/2011.04006/) (2020)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs](https://mlsanity.com/paper/1706.02633/) (2017)
- [ReZero is All You Need: Fast Convergence at Large Depth](https://mlsanity.com/paper/2003.04887/) (2020)
- [Synthetic Data -- what, why and how?](https://mlsanity.com/paper/2205.03257/) (2022)
- [Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Networks](https://mlsanity.com/paper/1908.02498/) (2019)
- [Unsupervised Brain Anomaly Detection and Segmentation with Transformers](https://mlsanity.com/paper/2102.11650/) (2021)
- [Transformer-based out-of-distribution detection for clinically safe segmentation](https://mlsanity.com/paper/2205.10650/) (2022)
- [Neuromorphologicaly-preserving Volumetric data encoding using VQ-VAE](https://mlsanity.com/paper/2002.05692/) (2020)

## Built on by (most cited first, 15 of 15)

- [Generative AI for Medical Imaging: extending the MONAI Framework](https://mlsanity.com/paper/2307.15208/) (2023)
- [Autoregressive Models in Vision: A Survey](https://mlsanity.com/paper/2411.05902/) (2024)
- [Unsupervised 3D out-of-distribution detection with latent diffusion models](https://mlsanity.com/paper/2307.03777/) (2023)
- [Generating 3D Brain Tumor Regions in MRI using Vector-Quantization Generative Adversarial Networks](https://mlsanity.com/paper/2310.01251/) (2023)
- [A 3D generative model of pathological multi-modal MR images and segmentations](https://mlsanity.com/paper/2311.04552/) (2023)
- [BS-LDM: Effective Bone Suppression in High-Resolution Chest X-Ray Images with Conditional Latent Diffusion Models](https://mlsanity.com/paper/2412.15670/) (2024)
- [Latent Pollution Model: The Hidden Carbon Footprint in 3D Image Synthesis](https://mlsanity.com/paper/2407.14892/) (2024)
- [Towards the Discovery of Down Syndrome Brain Biomarkers Using Generative Models](https://mlsanity.com/paper/2409.13437/) (2024)
- [ITCFN: Incomplete Triple-Modal Co-Attention Fusion Network for Mild Cognitive Impairment Conversion Prediction](https://mlsanity.com/paper/2501.11276/) (2025)
- [Exploring the Effectiveness of Deep Features from Domain-Specific Foundation Models in Retinal Image Synthesis](https://mlsanity.com/paper/2506.11753/) (2025)
- [T-CACE: A Time-Conditioned Autoregressive Contrast Enhancement Multi-Task Framework for Contrast-Free Liver MRI Synthesis, Segmentation, and Diagnosis](https://mlsanity.com/paper/2508.09919/) (2025)
- [Cycle Diffusion Model for Counterfactual Image Generation](https://mlsanity.com/paper/2509.24267/) (2025)
- [Beyond Representation Learning: A Systematic Study of Joint-Embedding Predictive Generation for 3D Brain MRI](https://mlsanity.com/paper/2608.28787/) (2026)
- [Generating Novel Brain Morphology by Deforming Learned Templates](https://mlsanity.com/paper/2503.03778/) (2025)
- [MRExtrap: Longitudinal Aging of Brain MRIs using Linear Modeling in Latent Space](https://mlsanity.com/paper/2508.19482/) (2025)

## Abstract (arXiv)

Human anatomy, morphology, and associated diseases can be studied using medical imaging data. However, access to medical imaging data is restricted by governance and privacy concerns, data ownership, and the cost of acquisition, thus limiting our ability to understand the human body. A possible solution to this issue is the creation of a model able to learn and then generate synthetic images of the human body conditioned on specific characteristics of relevance (e.g., age, sex, and disease status). Deep generative models, in the form of neural networks, have been recently used to create synthetic 2D images of natural scenes. Still, the ability to produce high-resolution 3D volumetric imaging data with correct anatomical morphology has been hampered by data scarcity and algorithmic and computational limitations. This work proposes a generative model that can be scaled to produce anatomically correct, high-resolution, and realistic images of the human brain, with the necessary quality to allow further downstream analyses. The ability to generate a potentially unlimited amount of data not only enables large-scale studies of human anatomy and pathology without jeopardizing patient privacy, but also significantly advances research in the field of anomaly detection, modality synthesis, learning under limited data, and fair and ethical AI. Code and trained models are available at: https://github.com/AmigoLab/SynthAnatomy.