# Portrait3D: Text-Guided High-Quality 3D Portrait Generation Using Pyramid Representation and GANs Prior arXiv: https://arxiv.org/abs/2404.10394 ยท HTML: https://mlsanity.com/paper/2404.10394/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. - Builds on 46 of its 60 references that are in the map. - First posted on arXiv 16 Apr 2024. ## Placed in - Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/) ## Builds on (most cited first, 30 of 46) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023) - [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022) - [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022) - [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017) - [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021) - [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017) - [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015) - [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021) - [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023) - [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020) - [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018) - [Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions](https://mlsanity.com/paper/2303.12789/) (2023) - [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021) - [GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images](https://mlsanity.com/paper/2209.11163/) (2022) - [Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion](https://mlsanity.com/paper/2212.06135/) (2022) - [LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching](https://mlsanity.com/paper/2311.11284/) (2023) - [BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images](https://mlsanity.com/paper/2002.08988/) (2020) - [VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids](https://mlsanity.com/paper/2206.07695/) (2022) - [Visual Object Networks: Image Generation with Disentangled 3D Representation](https://mlsanity.com/paper/1812.02725/) (2018) - [Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatars](https://mlsanity.com/paper/2211.11208/) (2022) - [TADA! Text to Animatable Digital Avatars](https://mlsanity.com/paper/2308.10899/) (2023) - [EpiGRAF: Rethinking training of 3D GANs](https://mlsanity.com/paper/2206.10535/) (2022) - [imGHUM: Implicit Generative Models of 3D Human Shape and Articulated Pose](https://mlsanity.com/paper/2108.10842/) (2021) - [DreamHuman: Animatable 3D Avatars from Text](https://mlsanity.com/paper/2306.09329/) (2023) - [HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting](https://mlsanity.com/paper/2311.17061/) (2023) - [MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatar](https://mlsanity.com/paper/2312.04558/) (2023) - [HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generation](https://mlsanity.com/paper/2310.01406/) (2023) - [AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Control](https://mlsanity.com/paper/2303.17606/) (2023) - [SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion](https://mlsanity.com/paper/2311.15855/) (2023) ## Built on by (most cited first, 15 of 15) - [Zero-1-to-A: Zero-Shot One Image to Animatable Head Avatars Using Video Diffusion](https://mlsanity.com/paper/2503.15851/) (2025) - [HyPlaneHead: Rethinking Tri-plane-like Representations in Full-Head Image Synthesis](https://mlsanity.com/paper/2509.16748/) (2025) - [A Comprehensive Taxonomy and Analysis of Talking Head Synthesis: Techniques for Portrait Generation, Driving Mechanisms, and Editing](https://mlsanity.com/paper/2406.10553/) (2024) - [Generative Human Geometry Distribution](https://mlsanity.com/paper/2503.01448/) (2025) - [FaceCraft4D: Animated 3D Facial Avatar Generation from a Single Image](https://mlsanity.com/paper/2504.15179/) (2025) - [Dream, Lift, Animate: From Single Images to Animatable Gaussian Avatars](https://mlsanity.com/paper/2507.15979/) (2025) - [Bringing Your Portrait to 3D Presence](https://mlsanity.com/paper/2511.22553/) (2025) - [Text-based Animatable 3D Avatars with Morphable Model Alignment](https://mlsanity.com/paper/2504.15835/) (2025) - [Human Geometry Distribution for 3D Animation Generation](https://mlsanity.com/paper/2512.07459/) (2025) - [Condition Matters in Full-head 3D GANs](https://mlsanity.com/paper/2602.07198/) (2026) - [Towards High-Fidelity 3D Portrait Generation with Rich Details by Cross-View Prior-Aware Diffusion](https://mlsanity.com/paper/2411.10369/) (2024) - [StyleMM: Stylized 3D Morphable Face Model via Text-Driven Aligned Image Translation](https://mlsanity.com/paper/2508.11203/) (2025) - [GenLCA: 3D Diffusion for Full-Body Avatars from In-the-Wild Videos](https://mlsanity.com/paper/2604.07273/) (2026) - [EmoteGPT: 3D Human Facial Expressions from Natural Language Descriptions](https://mlsanity.com/paper/2607.02674/) (2026) - [ArchComplete: Autoregressive 3D Architectural Design Generation with Hierarchical Diffusion-Based Upsampling](https://mlsanity.com/paper/2412.17957/) (2024) ## Abstract (arXiv) Existing neural rendering-based text-to-3D-portrait generation methods typically make use of human geometry prior and diffusion models to obtain guidance. However, relying solely on geometry information introduces issues such as the Janus problem, over-saturation, and over-smoothing. We present Portrait3D, a novel neural rendering-based framework with a novel joint geometry-appearance prior to achieve text-to-3D-portrait generation that overcomes the aforementioned issues. To accomplish this, we train a 3D portrait generator, 3DPortraitGAN-Pyramid, as a robust prior. This generator is capable of producing 360deg canonical 3D portraits, serving as a starting point for the subsequent diffusion-based generation process. To mitigate the "grid-like" artifact caused by the high-frequency information in the feature-map-based 3D representation commonly used by most 3D-aware GANs, we integrate a novel pyramid tri-grid 3D representation into 3DPortraitGAN-Pyramid. To generate 3D portraits from text, we first project a randomly generated image aligned with the given prompt into the pre-trained 3DPortraitGAN-Pyramid's latent space. The resulting latent code is then used to synthesize a pyramid tri-grid. Beginning with the obtained pyramid tri-grid, we use score distillation sampling to distill the diffusion model's knowledge into the pyramid tri-grid. Following that, we utilize the diffusion model to refine the rendered images of the 3D portrait and then use these refined images as training data to further optimize the pyramid tri-grid, effectively eliminating issues with unrealistic color and unnatural artifacts. Our experimental results show that Portrait3D can produce realistic, high-quality, and canonical 3D portraits that align with the prompt.