# Frequency-Controlled Diffusion Model for Versatile Text-Guided Image-to-Image Translation

arXiv: https://arxiv.org/abs/2407.03006 · HTML: https://mlsanity.com/paper/2407.03006/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 3 Jul 2024.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 28 of 28)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation](https://mlsanity.com/paper/2011.12799/) (2020)
- [Text2LIVE: Text-Driven Layered Image and Video Editing](https://mlsanity.com/paper/2204.02491/) (2022)
- [VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance](https://mlsanity.com/paper/2204.08583/) (2022)
- [DRIT++: Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1905.01270/) (2019)
- [Splicing ViT Features for Semantic Appearance Transfer](https://mlsanity.com/paper/2201.00424/) (2022)
- [Diffusion-based Image Translation using Disentangled Style and Content Representation](https://mlsanity.com/paper/2209.15264/) (2022)
- [Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping](https://mlsanity.com/paper/1809.05852/) (2018)
- [Frequency Domain Image Translation: More Photo-realistic, Better Identity-preserving](https://mlsanity.com/paper/2011.13611/) (2020)
- [Learning Frequency-aware Dynamic Network for Efficient Super-Resolution](https://mlsanity.com/paper/2103.08357/) (2021)
- [DCT-Conv: Coding filters in convolutional networks with Discrete Cosine Transform](https://mlsanity.com/paper/2001.08517/) (2020)

## Built on by (most cited first, 12 of 12)

- [DiffV2IR: Visible-to-Infrared Diffusion Model via Vision-Language Understanding](https://mlsanity.com/paper/2503.19012/) (2025)
- [FDS: Frequency-Aware Denoising Score for Text-Guided Latent Diffusion Image Editing](https://mlsanity.com/paper/2503.19191/) (2025)
- [PTDiffusion: Free Lunch for Generating Optical Illusion Hidden Pictures with Phase-Transferred Diffusion Model](https://mlsanity.com/paper/2503.06186/) (2025)
- [StyleMe3D: Stylization with Disentangled Priors by Multiple Encoders on 3D Gaussians](https://mlsanity.com/paper/2504.15281/) (2025)
- [FreqEdit: Preserving High-Frequency Features for Robust Multi-Turn Image Editing](https://mlsanity.com/paper/2512.01755/) (2025)
- [FBSDiff: Plug-and-Play Frequency Band Substitution of Diffusion Features for Highly Controllable Text-Driven Image Translation](https://mlsanity.com/paper/2408.00998/) (2024)
- [Exploring the Design Space of Diffusion Bridge Models](https://mlsanity.com/paper/2410.21553/) (2024)
- [Everything at Every Scale: Scale-Invariant Diffusion with Continuous Super-Resolution](https://mlsanity.com/paper/2605.26032/) (2026)
- [Token Painter: Training-Free Text-Guided Image Inpainting via Mask Autoregressive Models](https://mlsanity.com/paper/2509.23919/) (2025)
- [FRBNet: Revisiting Low-Light Vision through Frequency-Domain Radial Basis Network](https://mlsanity.com/paper/2510.23444/) (2025)
- [FourierSampler: Unlocking Non-Autoregressive Potential in Diffusion Language Models via Frequency-Guided Generation](https://mlsanity.com/paper/2601.23182/) (2026)
- [ADP-DiT: Text-Guided Diffusion Transformer for Brain Image Generation in Alzheimer's Disease Progression](https://mlsanity.com/paper/2604.13495/) (2026)

## Abstract (arXiv)

Recently, large-scale text-to-image (T2I) diffusion models have emerged as a powerful tool for image-to-image translation (I2I), allowing open-domain image translation via user-provided text prompts. This paper proposes frequency-controlled diffusion model (FCDiffusion), an end-to-end diffusion-based framework that contributes a novel solution to text-guided I2I from a frequency-domain perspective. At the heart of our framework is a feature-space frequency-domain filtering module based on Discrete Cosine Transform, which filters the latent features of the source image in the DCT domain, yielding filtered image features bearing different DCT spectral bands as different control signals to the pre-trained Latent Diffusion Model. We reveal that control signals of different DCT spectral bands bridge the source image and the T2I generated image in different correlations (e.g., style, structure, layout, contour, etc.), and thus enable versatile I2I applications emphasizing different I2I correlations, including style-guided content creation, image semantic manipulation, image scene translation, and image style translation. Different from related approaches, FCDiffusion establishes a unified text-guided I2I framework suitable for diverse image translation tasks simply by switching among different frequency control branches at inference time. The effectiveness and superiority of our method for text-guided I2I are demonstrated with extensive experiments both qualitatively and quantitatively. Our project is publicly available at: https://xianggao1102.github.io/FCDiffusion/.