# Memory-efficient GAN-based Domain Translation of High Resolution 3D Medical Images

arXiv: https://arxiv.org/abs/2010.03396 · HTML: https://mlsanity.com/paper/2010.03396/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2020.

## Built on by (most cited first, 14 of 14)

- [Make-A-Volume: Leveraging Latent Diffusion Models for Cross-Modality 3D Brain MRI Synthesis](https://mlsanity.com/paper/2307.10094/) (2023)
- [Deep Generative Models for 3D Medical Image Synthesis](https://mlsanity.com/paper/2410.17664/) (2024)
- [Synthesis-based Imaging-Differentiation Representation Learning for Multi-Sequence 3D/4D MRI](https://mlsanity.com/paper/2302.00517/) (2023)
- [An Explainable Deep Framework: Towards Task-Specific Fusion for Multi-to-One MRI Synthesis](https://mlsanity.com/paper/2307.00885/) (2023)
- [A Data Augmentation Pipeline to Generate Synthetic Labeled Datasets of 3D Echocardiography Images using a GAN](https://mlsanity.com/paper/2403.05384/) (2024)
- [A Domain Translation Framework with an Adversarial Denoising Diffusion Model to Generate Synthetic Datasets of Echocardiography Images](https://mlsanity.com/paper/2403.04612/) (2024)
- [McCaD: Multi-Contrast MRI Conditioned, Adaptive Adversarial Diffusion Model for High-Fidelity MRI Synthesis](https://mlsanity.com/paper/2409.00585/) (2024)
- [Uncertainty-Aware ControlNet: Bridging Domain Gaps with Synthetic Image Generation](https://mlsanity.com/paper/2510.11346/) (2025)
- [ContourDiff: Unpaired Medical Image Translation with Structural Consistency](https://mlsanity.com/paper/2403.10786/) (2024)
- [LPUWF-LDM: Enhanced Latent Diffusion Model for Precise Late-phase UWF-FA Generation on Limited Dataset](https://mlsanity.com/paper/2409.00726/) (2024)
- [Causal Image Synthesis of Brain MR in 3D](https://mlsanity.com/paper/2303.14349/) (2023)
- [Domain Transfer Through Image-to-Image Translation for Uncertainty-Aware Prostate Cancer Classification](https://mlsanity.com/paper/2307.00479/) (2023)
- [Bringing together invertible UNets with invertible attention modules for memory-efficient diffusion models](https://mlsanity.com/paper/2504.10883/) (2025)
- [Structure-Adaptive Sparse Diffusion in Voxel Space for 3D Medical Image Enhancement](https://mlsanity.com/paper/2604.17773/) (2026)

## Abstract (arXiv)

Generative adversarial networks (GANs) are currently rarely applied on 3D medical images of large size, due to their immense computational demand. The present work proposes a multi-scale patch-based GAN approach for establishing unpaired domain translation by generating 3D medical image volumes of high resolution in a memory-efficient way. The key idea to enable memory-efficient image generation is to first generate a low-resolution version of the image followed by the generation of patches of constant sizes but successively growing resolutions. To avoid patch artifacts and incorporate global information, the patch generation is conditioned on patches from previous resolution scales. Those multi-scale GANs are trained to generate realistically looking images from image sketches in order to perform an unpaired domain translation. This allows to preserve the topology of the test data and generate the appearance of the training domain data. The evaluation of the domain translation scenarios is performed on brain MRIs of size 155x240x240 and thorax CTs of size up to 512x512x512. Compared to common patch-based approaches, the multi-resolution scheme enables better image quality and prevents patch artifacts. Also, it ensures constant GPU memory demand independent from the image size, allowing for the generation of arbitrarily large images.