# Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representations

arXiv: https://arxiv.org/abs/2204.08839 · HTML: https://mlsanity.com/paper/2204.08839/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 60 of its 77 references that are in the map.
- First posted on arXiv 19 Apr 2022.

## Builds on (most cited first, 30 of 60)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [FastNeRF: High-Fidelity Neural Rendering at 200FPS](https://mlsanity.com/paper/2103.10380/) (2021)
- [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)

## Built on by (most cited first, 30 of 65)

- [ECON: Explicit Clothed humans Optimized via Normal integration](https://mlsanity.com/paper/2212.07422/) (2022)
- [DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models](https://mlsanity.com/paper/2304.00916/) (2023)
- [InstantAvatar: Learning Avatars from Monocular Video in 60 Seconds](https://mlsanity.com/paper/2212.10550/) (2022)
- [Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatars](https://mlsanity.com/paper/2211.11208/) (2022)
- [EVA3D: Compositional 3D Human Generation from 2D Image Collections](https://mlsanity.com/paper/2210.04888/) (2022)
- [TeCH: Text-guided Reconstruction of Lifelike Clothed Humans](https://mlsanity.com/paper/2308.08545/) (2023)
- [Generative Neural Articulated Radiance Fields](https://mlsanity.com/paper/2206.14314/) (2022)
- [PanoHead: Geometry-Aware 3D Full-Head Synthesis in 360circ](https://mlsanity.com/paper/2303.13071/) (2023)
- [Advances in 3D Generation: A Survey](https://mlsanity.com/paper/2401.17807/) (2024)
- [Gaussian Shell Maps for Efficient 3D Human Generation](https://mlsanity.com/paper/2311.17857/) (2023)
- [Neural Parametric Gaussians for Monocular Non-Rigid Object Reconstruction](https://mlsanity.com/paper/2312.01196/) (2023)
- [AvatarGen: A 3D Generative Model for Animatable Human Avatars](https://mlsanity.com/paper/2211.14589/) (2022)
- [AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars](https://mlsanity.com/paper/2205.08535/) (2022)
- [Fast-SNARF: A Fast Deformer for Articulated Neural Fields](https://mlsanity.com/paper/2211.15601/) (2022)
- [AG3D: Learning to Generate 3D Avatars from 2D Image Collections](https://mlsanity.com/paper/2305.02312/) (2023)
- [Learning Locally Editable Virtual Humans](https://mlsanity.com/paper/2305.00121/) (2023)
- [PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations](https://mlsanity.com/paper/2404.04421/) (2024)
- [Global-correlated 3D-decoupling Transformer for Clothed Avatar Reconstruction](https://mlsanity.com/paper/2309.13524/) (2023)
- [PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation](https://mlsanity.com/paper/2312.04559/) (2023)
- [HumanGen: Generating Human Radiance Fields with Explicit Priors](https://mlsanity.com/paper/2212.05321/) (2022)
- [Disentangled Clothed Avatar Generation from Text Descriptions](https://mlsanity.com/paper/2312.05295/) (2023)
- [Human101: Training 100+FPS Human Gaussians in 100s from 1 View](https://mlsanity.com/paper/2312.15258/) (2023)
- [TriHuman : A Real-time and Controllable Tri-plane Representation for Detailed Human Geometry and Appearance Synthesis](https://mlsanity.com/paper/2312.05161/) (2023)
- [Recent Trends in 3D Reconstruction of General Non-Rigid Scenes](https://mlsanity.com/paper/2403.15064/) (2024)
- [GETAvatar: Generative Textured Meshes for Animatable Human Avatars](https://mlsanity.com/paper/2310.02714/) (2023)
- [UrbanGIRAFFE: Representing Urban Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2303.14167/) (2023)
- [NPC: Neural Point Characters from Video](https://mlsanity.com/paper/2304.02013/) (2023)
- [TOSS:High-quality Text-guided Novel View Synthesis from a Single Image](https://mlsanity.com/paper/2310.10644/) (2023)
- [HumanLiff: Layer-wise 3D Human Generation with Diffusion Model](https://mlsanity.com/paper/2308.09712/) (2023)
- [3DHumanGAN: 3D-Aware Human Image Generation with 3D Pose Mapping](https://mlsanity.com/paper/2212.07378/) (2022)

## Abstract (arXiv)

We propose an unsupervised method for 3D geometry-aware representation learning of articulated objects, in which no image-pose pairs or foreground masks are used for training. Though photorealistic images of articulated objects can be rendered with explicit pose control through existing 3D neural representations, these methods require ground truth 3D pose and foreground masks for training, which are expensive to obtain. We obviate this need by learning the representations with GAN training. The generator is trained to produce realistic images of articulated objects from random poses and latent vectors by adversarial training. To avoid a high computational cost for GAN training, we propose an efficient neural representation for articulated objects based on tri-planes and then present a GAN-based framework for its unsupervised training. Experiments demonstrate the efficiency of our method and show that GAN-based training enables the learning of controllable 3D representations without paired supervision.