mlsanityData to 7 Oct 2026

BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images

arXiv 2002.08988 ↗First posted 20 Feb 2020cs.CVcited by 166

Not on the map. Why?

Cited by 21 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present BlockGAN, an image generative model that learns object-aware 3D scene representations directly from unlabelled 2D images. Current work on scene representation learning either ignores scene background or treats the whole scene as one object. Meanwhile, work that considers scene compositionality treats scene objects only as image patches or 2D layers with alpha maps. Inspired by the computer graphics pipeline, we design BlockGAN to learn to first generate 3D features of background and foreground objects, then combine them into 3D features for the wholes cene, and finally render them into realistic images. This allows BlockGAN to reason over occlusion and interaction between objects' appearance, such as shadow and lighting, and provides control over each object's 3D pose and identity, while maintaining image realism. BlockGAN is trained end-to-end, using only unlabelled single images, without the need for 3D geometry, pose labels, object masks, or multiple views of the same scene. Our experiments show that using explicit 3D features to represent objects allows BlockGAN to learn disentangled representations both in terms of objects (foreground and background) and their properties (pose and identity).

Key facts. Cited by 166 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 32; 2022: 46; 2023: 48; 2024: 27; 2025: 7; 2026: 6.

166 papers.

Sort:

By area:

  1. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  2. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  3. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  4. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  5. 2021GAN Inversion: A Surveycited by 366
  6. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  7. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  8. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  9. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  10. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  11. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  12. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generationcited by 162
38 more
  1. 2021Illiterate DALL-E Learns to Composecited by 147
  2. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  3. 2021Generative Adversarial Transformerscited by 120
  4. 2021NeRF-VAE: A Geometry Aware 3D Scene Generative Modelcited by 111
  5. 2021GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinementcited by 109
  6. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Imagescited by 106
  7. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  8. 2022Generative Neural Articulated Radiance Fieldscited by 101
  9. 2021GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worldscited by 98
  10. 2021Decomposing 3D Scenes into Objects via Unsupervised Volume Segmentationcited by 88
  11. 2022ReLU Fields: The Little Non-linearity That Couldcited by 87
  12. 2022GIRAFFE HD: A High-Resolution 3D-aware Generative Modelcited by 86
  13. 2022Object-Compositional Neural Implicit Surfacescited by 74
  14. 2022GRAM-HD: 3D-Consistent Image Generation at High Resolution with Generative Radiance Manifoldscited by 73
  15. 2021A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesiscited by 72
  16. 2022Texturify: Generating Textures on 3D Shape Surfacescited by 65
  17. 2022Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representationscited by 65
  18. 2022High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimizationcited by 65
  19. 2021ACORN: Adaptive Coordinate Networks for Neural Scene Representationcited by 64
  20. 2024Advances in 3D Generation: A Surveycited by 62
  21. 2024SPAD : Spatially Aware Multiview Diffuserscited by 62
  22. 2021DOVE: Learning Deformable 3D Objects by Watching Videoscited by 61
  23. 2022DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesiscited by 61
  24. 2022AvatarGen: A 3D Generative Model for Animatable Human Avatarscited by 59
  25. 2023Diffusion Handles: Enabling 3D Edits for Diffusion Models by Lifting Activations to 3Dcited by 59
  26. 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
  27. 2023CC3D: Layout-Conditioned Generation of Compositional 3D Scenescited by 58
  28. 2022Generative Multiplane Images: Making a 2D GAN 3D-Awarecited by 57
  29. 2021CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fieldscited by 55
  30. 20223D GAN Inversion for Controllable Portrait Image Animationcited by 53
  31. 2021Generative Occupancy Fields for 3D Surface-Aware Image Synthesiscited by 51
  32. 2023AutoDecoding Latent 3D Diffusion Modelscited by 50
  33. 2022AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatarscited by 46
  34. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Modelcited by 45
  35. 2022Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesiscited by 43
  36. 2023TriPlaneNet: An Encoder for EG3D Inversioncited by 41
  37. 2022Disentangled3D: Learning a 3D Generative Model with Disentangled Geometry and Appearance from Monocular Imagescited by 39
  38. 2022VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relightingcited by 37

50 most cited of 166 All 166 (JSON)