mlsanityData to 7 Oct 2026

Diffusion Guided Domain Adaptation of Image Generators

arXiv 2212.04473 ↗First posted 8 Dec 2022cs.CVcited by 35

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Can a text-to-image diffusion model be used as a training objective for adapting a GAN generator to another domain? In this paper, we show that the classifier-free guidance can be leveraged as a critic and enable generators to distill knowledge from large-scale text-to-image diffusion models. Generators can be efficiently shifted into new domains indicated by text prompts without access to groundtruth samples from target domains. We demonstrate the effectiveness and controllability of our method through extensive experiments. Although not trained to minimize CLIP loss, our model achieves equally high CLIP scores and significantly lower FID than prior work on short prompts, and outperforms the baseline qualitatively and quantitatively on long and complicated prompts. To our best knowledge, the proposed method is the first attempt at incorporating large-scale pre-trained diffusion models and distillation sampling for text-driven image generator domain adaptation and gives a quality previously beyond possible. Moreover, we extend our work to 3D-aware style-based generators and DreamBooth guidance.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 41 of its 59 references that are in the map. First posted on arXiv 8 Dec 2022.

Builds on

41 of 59 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  5. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation6 Oct

Other references · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  12. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
24 more
  1. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  2. 2022Video Diffusion Modelscited by 2,280
  3. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  4. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  5. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  6. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  7. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  8. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  9. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  10. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  11. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  12. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 447
  13. 2021Few-shot Image Generation via Cross-domain Correspondencecited by 206
  14. 2021Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesiscited by 140
  15. 2020Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANscited by 132
  16. 2020Few-shot Image Generation with Elastic Weight Consolidationcited by 122
  17. 2019MineGAN: effective knowledge transfer from GANs to target domains with few imagescited by 106
  18. 2021Regularizing Generative Adversarial Networks under Limited Datacited by 105
  19. 2019Image Generation From Small Datasets via Batch Statistics Adaptationcited by 102
  20. 2020Resolution Dependent GAN Interpolation for Controllable Image Synthesis Between Domainscited by 96
  21. 2021FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimizationcited by 71
  22. 2020Few-Shot Adaptation of Generative Adversarial Networkscited by 64
  23. 2021Data-Efficient Instance Generation from Instance Discriminationcited by 58
  24. 2021On Aliased Resizing and Surprising Subtleties in GAN Evaluationcited by 51

Built on by

Cited by year

Citing papers per year: 2023: 17; 2024: 14; 2025: 4.

35 papers.

Sort:

By area:

  1. 2023Delta Denoising Scorecited by 120
  2. 2023Improving Tuning-Free Real Image Editing with Proximal Guidancecited by 87
  3. 2023AVID: Any-Length Video Inpainting with Diffusion Modelcited by 81
  4. 2024You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scalecited by 65
  5. 2023Word-As-Image for Semantic Typographycited by 62
  6. 2024CoCoCo: Improving Text-Guided Video Inpainting for Better Consistency, Controllability and Compatibilitycited by 49
  7. 2024PALP: Prompt Aligned Personalization of Text-to-Image Modelscited by 35
  8. 2025DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimizationcited by 27
  9. 2023StyleAvatar3D: Leveraging Image-Text Diffusion Models for High-Fidelity 3D Avatar Generationcited by 25
  10. 2023Collaborative Score Distillation for Consistent Visual Synthesiscited by 23
  11. 2023Unsupervised Domain Adaptation via Domain-Adaptive Diffusioncited by 19
  12. 2023DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaptation by Combining 3D GANs and Diffusion Priorscited by 19
23 more
  1. 2023Adversarial Score Distillation: When score distillation meets GANcited by 15
  2. 2023Free-Editor: Zero-shot Text-driven 3D Scene Editingcited by 14
  3. 2024UniPaint: Unified Space-time Video Inpainting via Mixture-of-Expertscited by 13
  4. 2023PODIA-3D: Domain Adaptation of 3D Generative Model Across Large Domain Gap Using Pose-Preserved Text-to-Image Diffusioncited by 12
  5. 2024Transfer Learning for Diffusion Modelscited by 10
  6. 2023Image Synthesis under Limited Data: A Survey and Taxonomycited by 9
  7. 2023DeformToon3D: Deformable 3D Toonification from Neural Radiance Fieldscited by 9
  8. 2024Reference-Based 3D-Aware Image Editing with Triplanescited by 9
  9. 2023FaceDNeRF: Semantics-Driven Face Reconstruction, Prompt Editing and Relighting with Diffusion Modelscited by 7
  10. 2023Text-driven Visual Synthesis with Latent Diffusion Priorcited by 6
  11. 2023Warping the Residuals for Image Editing with StyleGANcited by 6
  12. 2023Efficient Text-Guided 3D-Aware Portrait Generation with Score Distillation Sampling on Distributioncited by 4
  13. 2024Foodfusion: A Novel Approach for Food Image Composition via Diffusion Modelscited by 4
  14. 2024GCA-3D: Towards Generalized and Consistent Domain Adaptation of 3D Generatorscited by 4
  15. 2024One-shot Generative Domain Adaptation in 3D GANscited by 3
  16. 2024FED-NeRF: Achieve High 3D Consistency and Temporal Coherence for Face Video Editing on Dynamic NeRFcited by 2
  17. 20243D-aware Image Generation and Editing with Multi-modal Conditionscited by 2
  18. 2024Identity Preserving 3D Head Stylization with Multiview Score Distillationcited by 2
  19. 2025DualNeRF: Text-Driven 3D Scene Editing via Dual-Field Representationcited by 2
  20. 2024UniHDA: A Unified and Versatile Framework for Multi-Modal Hybrid Domain Adaptationcited by 0
  21. 2024Generative Visual Communication in the Era of Vision-Language Modelscited by 0
  22. 2025FreeDNA: Endowing Domain Adaptation of Diffusion-Based Dense Prediction with Training-Free Domain Noise Alignmentcited by 0
  23. 2025StyleFusion360: View-Consistent Head Stylization via Adaptive Style Modulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Uni-DAD: Unified Distillation and Adaptation of Diffusion Models for Few-step Few-shot Image Generation
  2. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation
  3. 2024Source-Free Domain Adaptation with Diffusion-Guided Source Data Generation
  4. 2024E2GAN: Efficient Training of Efficient GANs for Image-to-Image Translation
  5. 2024GANFusion: Feed-Forward Text-to-3D with Diffusion in GAN Space
  6. 2024Training-free Composite Scene Generation for Layout-to-Image Synthesis
  7. 2023Zero-shot Generative Model Adaptation via Image-specific Prompt Learning
  8. 2022DATID-3D: Diversity-Preserved Domain Adaptation Using Text-to-Image Diffusion for 3D Generative Model