# Zero-shot Medical Image Translation via Frequency-Guided Diffusion Models

arXiv: https://arxiv.org/abs/2304.02742 · HTML: https://mlsanity.com/paper/2304.02742/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 5 Apr 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 23 of 23)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [Tackling the Generative Learning Trilemma with Denoising Diffusion GANs](https://mlsanity.com/paper/2112.07804/) (2021)
- [Unsupervised Medical Image Translation with Adversarial Diffusion Models](https://mlsanity.com/paper/2207.08208/) (2022)
- [EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations](https://mlsanity.com/paper/2207.06635/) (2022)
- [Low Dose CT Image Denoising Using a Generative Adversarial Network with Wasserstein Distance and Perceptual Loss](https://mlsanity.com/paper/1708.00961/) (2017)
- [Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping](https://mlsanity.com/paper/1809.05852/) (2018)
- [Breaking the Dilemma of Medical Image-to-image Translation](https://mlsanity.com/paper/2110.06465/) (2021)
- [Conversion Between CT and MRI Images Using Diffusion and Score-Matching Models](https://mlsanity.com/paper/2209.12104/) (2022)
- [Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance](https://mlsanity.com/paper/2210.05559/) (2022)
- [ADN: Artifact Disentanglement Network for Unsupervised Metal Artifact Reduction](https://mlsanity.com/paper/1908.01104/) (2019)
- [Non-saturating GAN training as divergence minimization](https://mlsanity.com/paper/2010.08029/) (2020)
- [Multitask 3D CBCT-to-CT Translation and Organs-at-Risk Segmentation Using Physics-Based Data Augmentation](https://mlsanity.com/paper/2103.05690/) (2021)

## Built on by (most cited first, 30 of 40)

- [Diffusion Models in Low-Level Vision: A Survey](https://mlsanity.com/paper/2406.11138/) (2024)
- [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023)
- [Conditional Image Synthesis with Diffusion Models: A Survey](https://mlsanity.com/paper/2409.19365/) (2024)
- [DiffBoost: Enhancing Medical Image Segmentation via Text-Guided Diffusion Model](https://mlsanity.com/paper/2310.12868/) (2023)
- [I-MedSAM: Implicit Medical Image Segmentation with Segment Anything](https://mlsanity.com/paper/2311.17081/) (2023)
- [Fr'echet Radiomic Distance (FRD): A Versatile Metric for Comparing Medical Imaging Datasets](https://mlsanity.com/paper/2412.01496/) (2024)
- [FreeTumor: Advance Tumor Segmentation via Large-Scale Tumor Synthesis](https://mlsanity.com/paper/2406.01264/) (2024)
- [SimuScope: Realistic Endoscopic Synthetic Dataset Generation through Surgical Simulation and Diffusion Models](https://mlsanity.com/paper/2412.02332/) (2024)
- [FeRA: Frequency-Energy Constrained Routing for Effective Diffusion Adaptation Fine-Tuning](https://mlsanity.com/paper/2511.17979/) (2025)
- [FDDM: Unsupervised Medical Image Translation with a Frequency-Decoupled Diffusion Model](https://mlsanity.com/paper/2311.12070/) (2023)
- [Applications of Large Models in Medicine](https://mlsanity.com/paper/2502.17132/) (2025)
- [Improving Robustness and Reliability in Medical Image Classification with Latent-Guided Diffusion and Nested-Ensembles](https://mlsanity.com/paper/2310.15952/) (2023)
- [Angio-Diff: Learning a Self-Supervised Adversarial Diffusion Model for Angiographic Geometry Generation](https://mlsanity.com/paper/2506.19455/) (2025)
- [Denoising Diffusion Probabilistic Models for Magnetic Resonance Fingerprinting](https://mlsanity.com/paper/2410.23318/) (2024)
- [Conditional Diffusion Models for CT Image Synthesis from CBCT: A Systematic Review](https://mlsanity.com/paper/2509.17790/) (2025)
- [Adaptive Domain Shift in Diffusion Models for Cross-Modality Image Translation](https://mlsanity.com/paper/2601.18623/) (2026)
- [SP-DiffDose: A Conditional Diffusion Model for Radiation Dose Prediction Based on Multi-Scale Fusion of Anatomical Structures, Guided by SwinTransformer and Projector](https://mlsanity.com/paper/2312.06187/) (2023)
- [ContourDiff: Unpaired Medical Image Translation with Structural Consistency](https://mlsanity.com/paper/2403.10786/) (2024)
- [CSIM: A Copula-based similarity index sensitive to local changes for Image quality assessment](https://mlsanity.com/paper/2410.01411/) (2024)
- [HC3L-Diff: Hybrid conditional latent diffusion with high frequency enhancement for CBCT-to-CT synthesis](https://mlsanity.com/paper/2411.01575/) (2024)
- [G2PDiffusion: Cross-Species Genotype-to-Phenotype Prediction via Evolutionary Diffusion](https://mlsanity.com/paper/2502.04684/) (2025)
- [Bi-level Guided Diffusion Models for Zero-Shot Medical Imaging Inverse Problems](https://mlsanity.com/paper/2404.03706/) (2024)
- [QID2: An Image-Conditioned Diffusion Model for Q-space Up-sampling of DWI Data](https://mlsanity.com/paper/2409.02309/) (2024)
- [SinoSynth: A Physics-based Domain Randomization Approach for Generalizable CBCT Image Enhancement](https://mlsanity.com/paper/2409.18355/) (2024)
- [Computationally Efficient Diffusion Models in Medical Imaging: A Comprehensive Review](https://mlsanity.com/paper/2505.07866/) (2025)
- [Uncertainty-Aware Distribution-to-Distribution Flow Matching for Scientific Imaging](https://mlsanity.com/paper/2603.21717/) (2026)
- [Multiple Noises in Diffusion Model for Semi-Supervised Multi-Domain Translation](https://mlsanity.com/paper/2309.14394/) (2023)
- [Prior Frequency Guided Diffusion Model for Limited Angle (LA)-CBCT Reconstruction](https://mlsanity.com/paper/2404.01448/) (2024)
- [HepatoGEN: Generating Hepatobiliary Phase MRI with Perceptual and Adversarial Models](https://mlsanity.com/paper/2504.18405/) (2025)
- [A Prior-Guided Joint Diffusion Model in Projection Domain for PET Tracer Conversion](https://mlsanity.com/paper/2506.16733/) (2025)

## Abstract (arXiv)

Recently, the diffusion model has emerged as a superior generative model that can produce high quality and realistic images. However, for medical image translation, the existing diffusion models are deficient in accurately retaining structural information since the structure details of source domain images are lost during the forward diffusion process and cannot be fully recovered through learned reverse diffusion, while the integrity of anatomical structures is extremely important in medical images. For instance, errors in image translation may distort, shift, or even remove structures and tumors, leading to incorrect diagnosis and inadequate treatments. Training and conditioning diffusion models using paired source and target images with matching anatomy can help. However, such paired data are very difficult and costly to obtain, and may also reduce the robustness of the developed model to out-of-distribution testing data. We propose a frequency-guided diffusion model (FGDM) that employs frequency-domain filters to guide the diffusion model for structure-preserving image translation. Based on its design, FGDM allows zero-shot learning, as it can be trained solely on the data from the target domain, and used directly for source-to-target domain translation without any exposure to the source-domain data during training. We evaluated it on three cone-beam CT (CBCT)-to-CT translation tasks for different anatomical sites, and a cross-institutional MR imaging translation task. FGDM outperformed the state-of-the-art methods (GAN-based, VAE-based, and diffusion-based) in metrics of Frechet Inception Distance (FID), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index Measure (SSIM), showing its significant advantages in zero-shot medical image translation.