# DiamondGAN: Unified Multi-Modal Generative Adversarial Networks for MRI Sequences Synthesis

arXiv: https://arxiv.org/abs/1904.12894 · HTML: https://mlsanity.com/paper/1904.12894/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2019.

## Built on by (most cited first, 30 of 36)

- [Unsupervised Medical Image Translation with Adversarial Diffusion Models](https://mlsanity.com/paper/2207.08208/) (2022)
- [ResViT: Residual vision transformers for multi-modal medical image synthesis](https://mlsanity.com/paper/2106.16031/) (2021)
- [CoLa-Diff: Conditional Latent Diffusion Model for Multi-Modal MRI Synthesis](https://mlsanity.com/paper/2303.14081/) (2023)
- [One Model to Synthesize Them All: Multi-contrast Multi-scale Transformer for Missing Data Imputation](https://mlsanity.com/paper/2204.13738/) (2022)
- [Unified Multi-Modal Image Synthesis for Missing Modality Imputation](https://mlsanity.com/paper/2304.05340/) (2023)
- [Self-Consistent Recursive Diffusion Bridge for Medical Image Translation](https://mlsanity.com/paper/2405.06789/) (2024)
- [I2I-Mamba: Multi-modal medical image synthesis via selective state space modeling](https://mlsanity.com/paper/2405.14022/) (2024)
- [The Brain Tumor Segmentation (BraTS) Challenge 2023: Brain MR Image Synthesis for Tumor Segmentation (BraSyn)](https://mlsanity.com/paper/2305.09011/) (2023)
- [One Model to Unite Them All: Personalized Federated Learning of Multi-Contrast MRI Synthesis](https://mlsanity.com/paper/2207.06509/) (2022)
- [Medical Image Segmentation on MRI Images with Missing Modalities: A Review](https://mlsanity.com/paper/2203.06217/) (2022)
- [Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging](https://mlsanity.com/paper/2107.09543/) (2021)
- [MRI Reconstruction with Side Information using Diffusion Models](https://mlsanity.com/paper/2303.14795/) (2023)
- [Unified Brain MR-Ultrasound Synthesis using Multi-Modal Hierarchical Representations](https://mlsanity.com/paper/2309.08747/) (2023)
- [Synthesis-based Imaging-Differentiation Representation Learning for Multi-Sequence 3D/4D MRI](https://mlsanity.com/paper/2302.00517/) (2023)
- [METGAN: Generative Tumour Inpainting and Modality Synthesis in Light Sheet Microscopy](https://mlsanity.com/paper/2104.10993/) (2021)
- [An Explainable Deep Framework: Towards Task-Specific Fusion for Multi-to-One MRI Synthesis](https://mlsanity.com/paper/2307.00885/) (2023)
- [Unified Cross-Modal Medical Image Synthesis with Hierarchical Mixture of Product-of-Experts](https://mlsanity.com/paper/2410.19378/) (2024)
- [Brain Multigraph Prediction using Topology-Aware Adversarial Graph Neural Network](https://mlsanity.com/paper/2105.02565/) (2021)
- [Cross-Modality Neuroimage Synthesis: A Survey](https://mlsanity.com/paper/2202.06997/) (2022)
- [Deep Quality Estimation: Creating Surrogate Models for Human Quality Ratings](https://mlsanity.com/paper/2205.10355/) (2022)
- [A Domain Translation Framework with an Adversarial Denoising Diffusion Model to Generate Synthetic Datasets of Echocardiography Images](https://mlsanity.com/paper/2403.04612/) (2024)
- [Learning Modality-Aware Representations: Adaptive Group-wise Interaction Network for Multimodal MRI Synthesis](https://mlsanity.com/paper/2411.14684/) (2024)
- [CoNeS: Conditional neural fields with shift modulation for multi-sequence MRI translation](https://mlsanity.com/paper/2309.03320/) (2023)
- [DS3-Net: Difficulty-perceived Common-to-T1ce Semi-Supervised Multimodal MRI Synthesis Network](https://mlsanity.com/paper/2203.06920/) (2022)
- [A Domain-specific Perceptual Metric via Contrastive Self-supervised Representation: Applications on Natural and Medical Images](https://mlsanity.com/paper/2212.01577/) (2022)
- [Resolution- and Stimulus-agnostic Super-Resolution of Ultra-High-Field Functional MRI: Application to Visual Studies](https://mlsanity.com/paper/2311.14918/) (2023)
- [DAMM-Diffusion: Learning Divergence-Aware Multi-Modal Diffusion Model for Nanoparticles Distribution Prediction](https://mlsanity.com/paper/2503.09491/) (2025)
- [MIST GAN: Modality Imputation Using Style Transfer for MRI](https://mlsanity.com/paper/2202.10396/) (2022)
- [Principled Feature Disentanglement for High-Fidelity Unified Brain MRI Synthesis](https://mlsanity.com/paper/2406.14954/) (2024)
- [Deterministic Medical Image Translation via High-fidelity Brownian Bridges](https://mlsanity.com/paper/2503.22531/) (2025)

## Abstract (arXiv)

Synthesizing MR imaging sequences is highly relevant in clinical practice, as single sequences are often missing or are of poor quality (e.g. due to motion). Naturally, the idea arises that a target modality would benefit from multi-modal input, as proprietary information of individual modalities can be synergistic. However, existing methods fail to scale up to multiple non-aligned imaging modalities, facing common drawbacks of complex imaging sequences. We propose a novel, scalable and multi-modal approach called DiamondGAN. Our model is capable of performing exible non-aligned cross-modality synthesis and data infill, when given multiple modalities or any of their arbitrary subsets, learning structured information in an end-to-end fashion. We synthesize two MRI sequences with clinical relevance (i.e., double inversion recovery (DIR) and contrast-enhanced T1 (T1-c)), reconstructed from three common sequences. In addition, we perform a multi-rater visual evaluation experiment and find that trained radiologists are unable to distinguish synthetic DIR images from real ones.