# AG3D: Learning to Generate 3D Avatars from 2D Image Collections

arXiv: https://arxiv.org/abs/2305.02312 · HTML: https://mlsanity.com/paper/2305.02312/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Builds on 61 of its 75 references that are in the map.
- First posted on arXiv 3 May 2023.

## Builds on (most cited first, 30 of 61)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video](https://mlsanity.com/paper/2201.04127/) (2022)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction](https://mlsanity.com/paper/2206.00665/) (2022)
- [Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans](https://mlsanity.com/paper/2110.04994/) (2021)
- [StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation](https://mlsanity.com/paper/2112.11427/) (2021)
- [ICON: Implicit Clothed humans Obtained from Normals](https://mlsanity.com/paper/2112.09127/) (2021)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)

## Built on by (most cited first, 30 of 55)

- [HUGS: Human Gaussian Splats](https://mlsanity.com/paper/2311.17910/) (2023)
- [TeCH: Text-guided Reconstruction of Lifelike Clothed Humans](https://mlsanity.com/paper/2308.08545/) (2023)
- [Hi3DGen: High-fidelity 3D Geometry Generation from Images via Normal Bridging](https://mlsanity.com/paper/2503.22236/) (2025)
- [Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Rendering](https://mlsanity.com/paper/2312.11360/) (2023)
- [MIMO: Controllable Character Video Synthesis with Spatial Decomposed Modeling](https://mlsanity.com/paper/2409.16160/) (2024)
- [Gaussian Shell Maps for Efficient 3D Human Generation](https://mlsanity.com/paper/2311.17857/) (2023)
- [PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations](https://mlsanity.com/paper/2404.04421/) (2024)
- [AniGS: Animatable Gaussian Avatar from a Single Image with Inconsistent Gaussian Reconstruction](https://mlsanity.com/paper/2412.02684/) (2024)
- [Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videos](https://mlsanity.com/paper/2403.17103/) (2024)
- [HumanLiff: Layer-wise 3D Human Generation with Diffusion Model](https://mlsanity.com/paper/2308.09712/) (2023)
- [Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation](https://mlsanity.com/paper/2401.04728/) (2024)
- [SuperGaussian: Repurposing Video Models for 3D Super Resolution](https://mlsanity.com/paper/2406.00609/) (2024)
- [DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaptation by Combining 3D GANs and Diffusion Priors](https://mlsanity.com/paper/2312.16837/) (2023)
- [XAGen: 3D Expressive Human Avatars Generation](https://mlsanity.com/paper/2311.13574/) (2023)
- [StructLDM: Structured Latent Diffusion for 3D Human Generation](https://mlsanity.com/paper/2404.01241/) (2024)
- [AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image Collections](https://mlsanity.com/paper/2309.02186/) (2023)
- [En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data](https://mlsanity.com/paper/2401.01173/) (2024)
- [Instant 3D Human Avatar Generation using Image Diffusion Models](https://mlsanity.com/paper/2406.07516/) (2024)
- [Efficient 3D Articulated Human Generation with Layered Surface Volumes](https://mlsanity.com/paper/2307.05462/) (2023)
- [HumanDreamer-X: Photorealistic Single-image Human Avatars Reconstruction via Gaussian Restoration](https://mlsanity.com/paper/2504.03536/) (2025)
- [What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs](https://mlsanity.com/paper/2401.02411/) (2024)
- [Human-Centric Foundation Models: Perception, Generation and Agentic Modeling](https://mlsanity.com/paper/2502.08556/) (2025)
- [Reference-Based 3D-Aware Image Editing with Triplanes](https://mlsanity.com/paper/2404.03632/) (2024)
- [3DPortraitGAN: Learning One-Quarter Headshot 3D GANs from a Single-View Portrait Dataset with Diverse Body Poses](https://mlsanity.com/paper/2307.14770/) (2023)
- [AttriHuman-3D: Editable 3D Human Avatar Generation with Attribute Decomposition and Indexing](https://mlsanity.com/paper/2312.02209/) (2023)
- [MoGA: 3D Generative Avatar Prior for Monocular Gaussian Avatar Reconstruction](https://mlsanity.com/paper/2507.23597/) (2025)
- [MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics](https://mlsanity.com/paper/2510.01619/) (2025)
- [PEGASUS: Personalized Generative 3D Avatars with Composable Attributes](https://mlsanity.com/paper/2402.10636/) (2024)
- [Disentangled Clothed Avatar Generation with Layered Representation](https://mlsanity.com/paper/2501.04631/) (2025)
- [Joint2Human: High-quality 3D Human Generation via Compact Spherical Embedding of 3D Joints](https://mlsanity.com/paper/2312.08591/) (2023)

## Abstract (arXiv)

While progress in 2D generative models of human appearance has been rapid, many applications require 3D avatars that can be animated and rendered. Unfortunately, most existing methods for learning generative models of 3D humans with diverse shape and appearance require 3D training data, which is limited and expensive to acquire. The key to progress is hence to learn generative models of 3D avatars from abundant unstructured 2D image collections. However, learning realistic and complete 3D appearance and geometry in this under-constrained setting remains challenging, especially in the presence of loose clothing such as dresses. In this paper, we propose a new adversarial generative model of realistic 3D people from 2D images. Our method captures shape and deformation of the body and loose clothing by adopting a holistic 3D generator and integrating an efficient and flexible articulation module. To improve realism, we train our model using multiple discriminators while also integrating geometric cues in the form of predicted 2D normal maps. We experimentally find that our method outperforms previous 3D- and articulation-aware methods in terms of geometry and appearance. We validate the effectiveness of our model and the importance of each component via systematic ablation studies.