# Unpaired Volumetric Harmonization of Brain MRI with Conditional Latent Diffusion

arXiv: https://arxiv.org/abs/2408.09315 · HTML: https://mlsanity.com/paper/2408.09315/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 29 of its 59 references that are in the map.
- First posted on arXiv 18 Aug 2024.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [Brain Imaging Generation with Latent Diffusion Models](https://mlsanity.com/paper/2209.07162/) (2022)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining](https://mlsanity.com/paper/2107.09559/) (2021)
- [Demystifying Neural Style Transfer](https://mlsanity.com/paper/1701.01036/) (2017)
- [Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis](https://mlsanity.com/paper/1601.04589/) (2016)
- [Controlling Perceptual Factors in Neural Style Transfer](https://mlsanity.com/paper/1611.07865/) (2016)
- [CoLa-Diff: Conditional Latent Diffusion Model for Multi-Modal MRI Synthesis](https://mlsanity.com/paper/2303.14081/) (2023)
- [Make-A-Volume: Leveraging Latent Diffusion Models for Cross-Modality 3D Brain MRI Synthesis](https://mlsanity.com/paper/2307.10094/) (2023)
- [HSR-Diff:Hyperspectral Image Super-Resolution via Conditional Diffusion Models](https://mlsanity.com/paper/2306.12085/) (2023)
- [InverseSR: 3D Brain MRI Super-Resolution Using a Latent Diffusion Model](https://mlsanity.com/paper/2308.12465/) (2023)
- [ImUnity: a generalizable VAE-GAN solution for multicenter MR image harmonization](https://mlsanity.com/paper/2109.06756/) (2021)
- [A Style-aware Discriminator for Controllable Image Translation](https://mlsanity.com/paper/2203.15375/) (2022)
- [Disentangling A Single MR Modality](https://mlsanity.com/paper/2205.04982/) (2022)
- [DomainATM: Domain Adaptation Toolbox for Medical Data Analysis](https://mlsanity.com/paper/2209.11890/) (2022)
- [Quantitative Metrics for Benchmarking Medical Image Harmonization](https://mlsanity.com/paper/2402.04426/) (2024)
- [Harmonizing Flows: Unsupervised MR harmonization based on normalizing flows](https://mlsanity.com/paper/2301.11551/) (2023)

## Built on by (most cited first, 5 of 5)

- [From 2D to 3D, Deep Learning-based Shape Reconstruction in Magnetic Resonance Imaging: A Review](https://mlsanity.com/paper/2510.01296/) (2025)
- [IHF-Harmony: Multi-Modality Magnetic Resonance Images Harmonization using Invertible Hierarchy Flow Model](https://mlsanity.com/paper/2602.21536/) (2026)
- [Anatomically Consistent Cross-Contrast Super-Resolution of Anisotropic Brain T2w MRI](https://mlsanity.com/paper/2608.08401/) (2026)
- [Task-Adaptive 3D Cross-Field MRI Translation via Field-Conditioned Content-Style Pretraining](https://mlsanity.com/paper/2608.09264/) (2026)
- [Frozen Site-Probe Accuracy Is Not a Standalone Harmonization Score: An Audit of Evaluation Practice for Multi-Site MRI](https://mlsanity.com/paper/2610.14140/) (2026)

## Abstract (arXiv)

Multi-site structural MRI is increasingly used in neuroimaging studies to diversify subject cohorts. However, combining MR images acquired from various sites/centers may introduce site-related non-biological variations. Retrospective image harmonization helps address this issue, but current methods usually perform harmonization on pre-extracted hand-crafted radiomic features, limiting downstream applicability. Several image-level approaches focus on 2D slices, disregarding inherent volumetric information, leading to suboptimal outcomes. To this end, we propose a novel 3D MRI Harmonization framework through Conditional Latent Diffusion (HCLD) by explicitly considering image style and brain anatomy. It comprises a generalizable 3D autoencoder that encodes and decodes MRIs through a 4D latent space, and a conditional latent diffusion model that learns the latent distribution and generates harmonized MRIs with anatomical information from source MRIs while conditioned on target image style. This enables efficient volume-level MRI harmonization through latent style translation, without requiring paired images from target and source domains during training. The HCLD is trained and evaluated on 4,158 T1-weighted brain MRIs from three datasets in three tasks, assessing its ability to remove site-related variations while retaining essential biological features. Qualitative and quantitative experiments suggest the effectiveness of HCLD over several state-of-the-arts