# Compositional Cross-Modality Translation via Whole-Volume Multitask Latent Flow Matching

arXiv: https://arxiv.org/abs/2608.08135 · HTML: https://mlsanity.com/paper/2608.08135/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 19 of its 24 references that are in the map.
- First posted on arXiv 8 Aug 2026.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 19 of 19)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [MAISI: Medical AI for Synthetic Imaging](https://mlsanity.com/paper/2409.11169/) (2024)
- [SynthRAD2023 Grand Challenge dataset: generating synthetic CT for radiotherapy](https://mlsanity.com/paper/2303.16320/) (2023)
- [Machine Learning with Multi-Site Imaging Data: An Empirical Study on the Impact of Scanner Effects](https://mlsanity.com/paper/1910.04597/) (2019)
- [SynthRAD2025 Grand Challenge dataset: generating synthetic CTs for radiotherapy](https://mlsanity.com/paper/2502.17609/) (2025)
- [Medical Image Segmentation with 3D Convolutional Neural Networks: A Survey](https://mlsanity.com/paper/2108.08467/) (2021)
- [XGeM: A Multi-Prompt Foundation Model for Multimodal Medical Data Generation](https://mlsanity.com/paper/2501.04614/) (2025)
- [Multi-resolution Guided 3D GANs for Medical Image Translation](https://mlsanity.com/paper/2412.00575/) (2024)
- [Any-to-Any Vision-Language Model for Multimodal X-ray Imaging and Radiological Report Generation](https://mlsanity.com/paper/2505.01091/) (2025)
- [Texture-Aware StarGAN for CT data harmonisation](https://mlsanity.com/paper/2503.15058/) (2025)
- [Sample-Aware Test-Time Adaptation for Medical Image-to-Image Translation](https://mlsanity.com/paper/2508.00766/) (2025)
- [Retrieval-Augmented Anatomical Guidance for Text-to-CT Generation](https://mlsanity.com/paper/2603.08305/) (2026)
- [Cross Modality Image Translation In Medical Imaging Using Generative Frameworks](https://mlsanity.com/paper/2605.13686/) (2026)

## Abstract (arXiv)

Cross-modality medical image translation can reduce the burden of multi-modal acquisitions, yet the field remains constrained by two coupled limitations: methods operate on 2D slices or 3D patches rather than whole volumes, and train a separate model for each translation task. Both stem from a single cause, the absence of a sufficiently strong volumetric prior, which forces generative models to learn anatomical appearance and cross-modality mapping simultaneously, an ill-posed problem at the scale of available paired datasets. We propose to decouple these objectives. A large-scale pretrained 3D variational autoencoder provides a compact latent representation of volumetric appearance, reducing translation to a conditional flow-matching problem. This compression makes whole-volume processing tractable, while a resolution-aware sampling strategy preserves native anatomical scale. We train a single model jointly across inter-modality (MRI→CT, CBCT→CT) and intra-modality (MRI→MRI) tasks over three multi-center datasets. Across all tasks, whole-volume processing outperforms its patch-based counterpart, and the multi-task model matches task-specific baselines while replacing N networks with one. Crucially, joint training unlocks capabilities inaccessible to task-specific approaches: zero-shot generalization to anatomical regions unseen during training, within 0.15 SSIM of the fully supervised model, and compositional cross-dataset translation along paths never directly supervised. These results suggest that combining a strong volumetric prior with multitask training is a scalable route toward synthesis systems that generalize beyond their training distribution. Code is available at https://github.com/arco-group/Whole-Volume-Latent-FM.