# HoloGAN: Unsupervised learning of 3D representations from natural images

arXiv: https://arxiv.org/abs/1904.01326 · HTML: https://mlsanity.com/paper/1904.01326/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2019.

## Built on by (most cited first, 30 of 59)

- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction](https://mlsanity.com/paper/2304.06714/) (2023)
- [Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields](https://mlsanity.com/paper/2305.11588/) (2023)
- [NeRF-VAE: A Geometry Aware 3D Scene Generative Model](https://mlsanity.com/paper/2104.00587/) (2021)
- [EVA3D: Compositional 3D Human Generation from 2D Image Collections](https://mlsanity.com/paper/2210.04888/) (2022)
- [3D-aware Image Synthesis via Learning Structural and Textural Representations](https://mlsanity.com/paper/2112.10759/) (2021)
- [Compositional 3D Scene Generation using Locally Conditioned Diffusion](https://mlsanity.com/paper/2303.12218/) (2023)
- [GAN-Control: Explicitly Controllable GANs](https://mlsanity.com/paper/2101.02477/) (2021)
- [InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Images](https://mlsanity.com/paper/2207.11148/) (2022)
- [DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting](https://mlsanity.com/paper/2404.06903/) (2024)
- [Removing Objects From Neural Radiance Fields](https://mlsanity.com/paper/2212.11966/) (2022)
- [SPAD : Spatially Aware Multiview Diffusers](https://mlsanity.com/paper/2402.05235/) (2024)
- [CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fields](https://mlsanity.com/paper/2103.17269/) (2021)
- [Multimodal Image Synthesis and Editing: The Generative AI Era](https://mlsanity.com/paper/2112.13592/) (2021)
- [VideoMV: Consistent Multi-View Generation Based on Large Video Generative Model](https://mlsanity.com/paper/2403.12010/) (2024)
- [Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields](https://mlsanity.com/paper/2206.08361/) (2022)
- [PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervision](https://mlsanity.com/paper/2303.09554/) (2023)
- [GeoLRM: Geometry-Aware Large Reconstruction Model for High-Quality 3D Gaussian Generation](https://mlsanity.com/paper/2406.15333/) (2024)
- [SphereHead: Stable 3D Full-head Synthesis with Spherical Tri-plane Representation](https://mlsanity.com/paper/2404.05680/) (2024)
- [Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Survey](https://mlsanity.com/paper/2507.14501/) (2025)
- [Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpainting](https://mlsanity.com/paper/2404.19758/) (2024)
- [Real3D: Scaling Up Large Reconstruction Models with Real-World Images](https://mlsanity.com/paper/2406.08479/) (2024)
- [Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videos](https://mlsanity.com/paper/2312.13604/) (2023)
- [CoCoNets: Continuous Contrastive 3D Scene Representations](https://mlsanity.com/paper/2104.03851/) (2021)
- [Consolidating Attention Features for Multi-view Image Editing](https://mlsanity.com/paper/2402.14792/) (2024)
- [3D-FM GAN: Towards 3D-Controllable Face Manipulation](https://mlsanity.com/paper/2208.11257/) (2022)
- [Fine Detailed Texture Learning for 3D Meshes with Generative Models](https://mlsanity.com/paper/2203.09362/) (2022)
- [Lightplane: Highly-Scalable Components for Neural 3D Fields](https://mlsanity.com/paper/2404.19760/) (2024)

## Abstract (arXiv)

We propose a novel generative adversarial network (GAN) for the task of unsupervised learning of 3D representations from natural images. Most generative models rely on 2D kernels to generate images and make few assumptions about the 3D world. These models therefore tend to create blurry images or artefacts in tasks that require a strong 3D understanding, such as novel-view synthesis. HoloGAN instead learns a 3D representation of the world, and to render this representation in a realistic manner. Unlike other GANs, HoloGAN provides explicit control over the pose of generated objects through rigid-body transformations of the learnt 3D features. Our experiments show that using explicit 3D features enables HoloGAN to disentangle 3D pose and identity, which is further decomposed into shape and appearance, while still being able to generate images with similar or higher visual quality than other generative models. HoloGAN can be trained end-to-end from unlabelled 2D images only. Particularly, we do not require pose labels, 3D shapes, or multiple views of the same objects. This shows that HoloGAN is the first generative model that learns 3D representations from natural images in an entirely unsupervised manner.