mlsanityData to 14 Oct 2026

Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesis

arXiv 1912.05237 ↗First posted 11 Dec 2019cs.CVcited by 104

Not on the map. Why?

Abstract (arXiv)

In recent years, Generative Adversarial Networks have achieved impressive results in photorealistic image synthesis. This progress nurtures hopes that one day the classical rendering pipeline can be replaced by efficient models that are learned directly from images. However, current image synthesis models operate in the 2D domain where disentangling 3D properties such as camera viewpoint or object pose is challenging. Furthermore, they lack an interpretable and controllable representation. Our key hypothesis is that the image generation process should be modeled in 3D space as the physical world surrounding us is intrinsically three-dimensional. We define the new task of 3D controllable image synthesis and propose an approach for solving it by reasoning both in 3D space and in the 2D image domain. We demonstrate that our model is able to disentangle latent 3D factors of simple multi-object scenes in an unsupervised fashion from raw images. Compared to pure 2D baselines, it allows for synthesizing scenes that are consistent wrt. changes in viewpoint or object pose. We further evaluate various 3D representations in terms of their usefulness for this challenging task.

Key facts. Cited by 104 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 22; 2022: 30; 2023: 31; 2024: 17; 2025: 3; 2026: 1.

104 papers.

Sort:

By area:

  1. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  2. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
  3. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
  4. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  5. 20223D Neural Field Generation using Triplane Diffusioncited by 274
  6. 2021Learning Object-Compositional Neural Radiance Field for Editable Scene Renderingcited by 228
  7. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  8. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstructioncited by 180
  9. 2021Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Renderingcited by 158
  10. 2021Illiterate DALL-E Learns to Composecited by 147
  11. 2022VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Gridscited by 133
  12. 2021Counterfactual Generative Networkscited by 111
38 more
  1. 2022Generative Neural Articulated Radiance Fieldscited by 101
  2. 2023Compositional 3D Scene Generation using Locally Conditioned Diffusioncited by 101
  3. 2021gradSim: Differentiable simulation for system identification and visuomotor controlcited by 100
  4. 2023NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Modelscited by 76
  5. 2022GRAM-HD: 3D-Consistent Image Generation at High Resolution with Generative Radiance Manifoldscited by 73
  6. 2021A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesiscited by 72
  7. 2023Learning Neural Volumetric Representations of Dynamic Humans in Minutescited by 70
  8. 2024Advances in 3D Generation: A Surveycited by 63
  9. 2022AvatarGen: A 3D Generative Model for Animatable Human Avatarscited by 59
  10. 2021CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fieldscited by 55
  11. 2023AG3D: Learning to Generate 3D Avatars from 2D Image Collectionscited by 55
  12. 2023CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusioncited by 54
  13. 2023Online Map Vectorization for Autonomous Driving: A Rasterization Perspectivecited by 54
  14. 20223D GAN Inversion for Controllable Portrait Image Animationcited by 53
  15. 2021Generative Occupancy Fields for 3D Surface-Aware Image Synthesiscited by 51
  16. 2021Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Renderingcited by 46
  17. 2022AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatarscited by 46
  18. 2023Control3D: Towards Controllable Text-to-3D Generationcited by 46
  19. 2022Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesiscited by 43
  20. 2023Text2Immersion: Generative Immersive Scene with 3D Gaussianscited by 43
  21. 2024L3DG: Latent 3D Gaussian Diffusioncited by 42
  22. 2022Disentangled3D: Learning a 3D Generative Model with Disentangled Geometry and Appearance from Monocular Imagescited by 39
  23. 2022VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relightingcited by 37
  24. 2022Controllable 3D Face Synthesis with Conditional Generative Occupancy Fieldscited by 37
  25. 2023PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervisioncited by 36
  26. 2024RodinHD: High-Fidelity 3D Avatar Generation with Diffusion Modelscited by 36
  27. 2023OmniAvatar: Geometry-Guided Controllable 3D Head Synthesiscited by 34
  28. 2022HumanGen: Generating Human Radiance Fields with Explicit Priorscited by 33
  29. 2022DATID-3D: Diversity-Preserved Domain Adaptation Using Text-to-Image Diffusion for 3D Generative Modelcited by 32
  30. 20213D-Aware Semantic-Guided Generative Model for Human Synthesiscited by 31
  31. 2024SphereHead: Stable 3D Full-head Synthesis with Spherical Tri-plane Representationcited by 30
  32. 2022Neural Novel Actor: Learning a Generalized Animatable Neural Representation for Human Actorscited by 28
  33. 2023GETAvatar: Generative Textured Meshes for Animatable Human Avatarscited by 27
  34. 2023UrbanGIRAFFE: Representing Urban Scenes as Compositional Generative Neural Feature Fieldscited by 26
  35. 2023GINA-3D: Learning to Generate Implicit Neural Assets in the Wildcited by 25
  36. 2024Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creationcited by 23
  37. 2022GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wildcited by 21
  38. 2023Slot-VAE: Object-Centric Scene Generation with Slot Attentioncited by 20

50 most cited of 104 All 104 (JSON)