# DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaptation by Combining 3D GANs and Diffusion Priors arXiv: https://arxiv.org/abs/2312.16837 ยท HTML: https://mlsanity.com/paper/2312.16837/ ## Key facts - Cited by 19 papers in the map (data through 14 Oct 2026). - Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. - Builds on 50 of its 56 references that are in the map. - First posted on arXiv 28 Dec 2023. ## Placed in - Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/) ## Builds on (most cited first, 30 of 50) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022) - [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023) - [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019) - [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022) - [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022) - [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016) - [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022) - [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023) - [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021) - [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023) - [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020) - [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020) - [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020) - [Zero-Shot Text-Guided Object Generation with Dream Fields](https://mlsanity.com/paper/2112.01455/) (2021) - [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021) - [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021) - [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021) - [TEXTure: Text-Guided Texturing of 3D Shapes](https://mlsanity.com/paper/2302.01721/) (2023) - [StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation](https://mlsanity.com/paper/2112.11427/) (2021) - [CLIP-Mesh: Generating textured meshes from text using pretrained image-text models](https://mlsanity.com/paper/2203.13333/) (2022) - [CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation](https://mlsanity.com/paper/2110.02624/) (2021) - [Text2Tex: Text-driven Texture Synthesis via Diffusion Models](https://mlsanity.com/paper/2303.11396/) (2023) - [DreamBooth3D: Subject-Driven Text-to-3D Generation](https://mlsanity.com/paper/2303.13508/) (2023) - [Escaping Plato's Cave: 3D Shape From Adversarial Rendering](https://mlsanity.com/paper/1811.11606/) (2018) - [DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models](https://mlsanity.com/paper/2304.00916/) (2023) - [CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis](https://mlsanity.com/paper/2110.09788/) (2021) ## Built on by (most cited first, 19 of 19) - [Deepfake Generation and Detection: A Benchmark and Survey](https://mlsanity.com/paper/2403.17881/) (2024) - [Knowledge-Aligned Counterfactual-Enhancement Diffusion Perception for Unsupervised Cross-Domain Visual Emotion Recognition](https://mlsanity.com/paper/2505.19694/) (2025) - [AvatarArtist: Open-Domain 4D Avatarization](https://mlsanity.com/paper/2503.19906/) (2025) - [Instant 3D Human Avatar Generation using Image Diffusion Models](https://mlsanity.com/paper/2406.07516/) (2024) - [Hunyuan3D Studio: End-to-End AI Pipeline for Game-Ready 3D Asset Generation](https://mlsanity.com/paper/2509.12815/) (2025) - [Diffusion Models in 3D Vision: A Survey](https://mlsanity.com/paper/2410.04738/) (2024) - [Reference-Based 3D-Aware Image Editing with Triplanes](https://mlsanity.com/paper/2404.03632/) (2024) - [Foodfusion: A Novel Approach for Food Image Composition via Diffusion Models](https://mlsanity.com/paper/2408.14135/) (2024) - [GCA-3D: Towards Generalized and Consistent Domain Adaptation of 3D Generators](https://mlsanity.com/paper/2412.15491/) (2024) - [Revealing Directions for Text-guided 3D Face Editing](https://mlsanity.com/paper/2410.04965/) (2024) - [Identity Preserving 3D Head Stylization with Multiview Score Distillation](https://mlsanity.com/paper/2411.13536/) (2024) - [SuperNeRF-GAN: A Universal 3D-Consistent Super-Resolution Framework for Efficient and Enhanced 3D-Aware Image Synthesis](https://mlsanity.com/paper/2501.06770/) (2025) - [GANFusion: Feed-Forward Text-to-3D with Diffusion in GAN Space](https://mlsanity.com/paper/2412.16717/) (2024) - [EasyCraft: A Robust and Efficient Framework for Automatic Avatar Crafting](https://mlsanity.com/paper/2503.01158/) (2025) - [UniView: Enhancing Novel View Synthesis From A Single Image By Unifying Reference Features](https://mlsanity.com/paper/2509.04932/) (2025) - [UniHDA: A Unified and Versatile Framework for Multi-Modal Hybrid Domain Adaptation](https://mlsanity.com/paper/2401.12596/) (2024) - [DivAvatar: Diverse 3D Avatar Generation with a Single Prompt](https://mlsanity.com/paper/2402.17292/) (2024) - [Pattern-Aware Diffusion Synthesis of fMRI/dMRI with Tissue and Microstructural Refinement](https://mlsanity.com/paper/2511.04963/) (2025) - [StyleFusion360: View-Consistent Head Stylization via Adaptive Style Modulation](https://mlsanity.com/paper/2511.22411/) (2025) ## Abstract (arXiv) Text-guided domain adaptation and generation of 3D-aware portraits find many applications in various fields. However, due to the lack of training data and the challenges in handling the high variety of geometry and appearance, the existing methods for these tasks suffer from issues like inflexibility, instability, and low fidelity. In this paper, we propose a novel framework DiffusionGAN3D, which boosts text-guided 3D domain adaptation and generation by combining 3D GANs and diffusion priors. Specifically, we integrate the pre-trained 3D generative models (e.g., EG3D) and text-to-image diffusion models. The former provides a strong foundation for stable and high-quality avatar generation from text. And the diffusion models in turn offer powerful priors and guide the 3D generator finetuning with informative direction to achieve flexible and efficient text-guided domain adaptation. To enhance the diversity in domain adaptation and the generation capability in text-to-avatar, we introduce the relative distance loss and case-specific learnable triplane respectively. Besides, we design a progressive texture refinement module to improve the texture quality for both tasks above. Extensive experiments demonstrate that the proposed framework achieves excellent results in both domain adaptation and text-to-avatar tasks, outperforming existing methods in terms of generation quality and efficiency. The project homepage is at https://younglbw.github.io/DiffusionGAN3D-homepage/.