# Disentangled Representation Learning in Cardiac Image Analysis

arXiv: https://arxiv.org/abs/1903.09467 · HTML: https://mlsanity.com/paper/1903.09467/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2019.

## Built on by (most cited first, 30 of 69)

- [Learning Disentangled Representations in the Imaging Domain](https://mlsanity.com/paper/2108.12043/) (2021)
- [Semi-supervised Meta-learning with Disentanglement for Domain-generalised Medical Image Segmentation](https://mlsanity.com/paper/2106.13292/) (2021)
- [Echocardiography Segmentation with Enforced Temporal Consistency](https://mlsanity.com/paper/2112.02102/) (2021)
- [Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization](https://mlsanity.com/paper/2102.06315/) (2021)
- [MT-UDA: Towards Unsupervised Cross-modality Medical Image Segmentation with Limited Source Labels](https://mlsanity.com/paper/2203.12454/) (2022)
- [Algorithm Fairness in AI for Medicine and Healthcare](https://mlsanity.com/paper/2110.00603/) (2021)
- [Deep Generative Model for Periodic Graphs](https://mlsanity.com/paper/2201.11932/) (2022)
- [FedDis: Disentangled Federated Learning for Unsupervised Brain Pathology Segmentation](https://mlsanity.com/paper/2103.03705/) (2021)
- [CDDSA: Contrastive Domain Disentanglement and Style Augmentation for Generalizable Medical Image Segmentation](https://mlsanity.com/paper/2211.12081/) (2022)
- [HACA3: A Unified Approach for Multi-site MR Image Harmonization](https://mlsanity.com/paper/2212.06065/) (2022)
- [Adversarial Continual Learning for Multi-Domain Hippocampal Segmentation](https://mlsanity.com/paper/2107.08751/) (2021)
- [Unified Brain MR-Ultrasound Synthesis using Multi-Modal Hierarchical Representations](https://mlsanity.com/paper/2309.08747/) (2023)
- [Disentangling A Single MR Modality](https://mlsanity.com/paper/2205.04982/) (2022)
- [Is attention all you need in medical image analysis? A review](https://mlsanity.com/paper/2307.12775/) (2023)
- [vMFNet: Compositionality Meets Domain-generalised Segmentation](https://mlsanity.com/paper/2206.14538/) (2022)
- [SE(3)-Equivariant and Noise-Invariant 3D Rigid Motion Tracking in Brain MRI](https://mlsanity.com/paper/2312.13534/) (2023)
- [Graph-based Unsupervised Disentangled Representation Learning via Multimodal Large Language Models](https://mlsanity.com/paper/2407.18999/) (2024)
- [Controllable cardiac synthesis via disentangled anatomy arithmetic](https://mlsanity.com/paper/2107.01748/) (2021)
- [Unified Cross-Modal Medical Image Synthesis with Hierarchical Mixture of Product-of-Experts](https://mlsanity.com/paper/2410.19378/) (2024)
- [Recent Advances in Fibrosis and Scar Segmentation from Cardiac MRI: A State-of-the-Art Review and Future Perspectives](https://mlsanity.com/paper/2106.15707/) (2021)
- [Re-using Adversarial Mask Discriminators for Test-time Training under Distribution Shifts](https://mlsanity.com/paper/2108.11926/) (2021)
- [SD-LayerNet: Semi-supervised retinal layer segmentation in OCT using disentangled representation with anatomical priors](https://mlsanity.com/paper/2207.00458/) (2022)
- [NAISR: A 3D Neural Additive Model for Interpretable Shape Representation](https://mlsanity.com/paper/2303.09234/) (2023)
- [GLOWin: A Flow-based Invertible Generative Framework for Learning Disentangled Feature Representations in Medical Images](https://mlsanity.com/paper/2103.10868/) (2021)
- [MouseGAN++: Unsupervised Disentanglement and Contrastive Representation for Multiple MRI Modalities Synthesis and Structural Segmentation of Mouse Brain](https://mlsanity.com/paper/2212.01825/) (2022)
- [Learn to Ignore: Domain Adaptation for Multi-Site MRI Analysis](https://mlsanity.com/paper/2110.06803/) (2021)
- [Subject-Specific Lesion Generation and Pseudo-Healthy Synthesis for Multiple Sclerosis Brain Images](https://mlsanity.com/paper/2208.02135/) (2022)
- [Medical Image Segmentation with Domain Adaptation: A Survey](https://mlsanity.com/paper/2311.01702/) (2023)
- [Decomposing Normal and Abnormal Features of Medical Images into Discrete Latent Codes for Content-Based Image Retrieval](https://mlsanity.com/paper/2103.12328/) (2021)
- [unORANIC: Unsupervised Orthogonalization of Anatomy and Image-Characteristic Features](https://mlsanity.com/paper/2308.15507/) (2023)

## Abstract (arXiv)

Typically, a medical image offers spatial information on the anatomy (and pathology) modulated by imaging specific characteristics. Many imaging modalities including Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) can be interpreted in this way. We can venture further and consider that a medical image naturally factors into some spatial factors depicting anatomy and factors that denote the imaging characteristics. Here, we explicitly learn this decomposed (disentangled) representation of imaging data, focusing in particular on cardiac images. We propose Spatial Decomposition Network (SDNet), which factorises 2D medical images into spatial anatomical factors and non-spatial modality factors. We demonstrate that this high-level representation is ideally suited for several medical image analysis tasks, such as semi-supervised segmentation, multi-task segmentation and regression, and image-to-image synthesis. Specifically, we show that our model can match the performance of fully supervised segmentation models, using only a fraction of the labelled images. Critically, we show that our factorised representation also benefits from supervision obtained either when we use auxiliary tasks to train the model in a multi-task setting (e.g. regressing to known cardiac indices), or when aggregating multimodal data from different sources (e.g. pooling together MRI and CT data). To explore the properties of the learned factorisation, we perform latent-space arithmetic and show that we can synthesise CT from MR and vice versa, by swapping the modality factors. We also demonstrate that the factor holding image specific information can be used to predict the input modality with high accuracy. Code will be made available at https://github.com/agis85/anatomy_modality_decomposition.