mlsanityData to 7 Oct 2026

3inGAN: Learning a 3D Generative Model from Images of a Self-similar Scene

arXiv 2211.14902 ↗First posted 27 Nov 2022cs.CV cs.GRcited by 9

Not on the map. Why?

Abstract (arXiv)

We introduce 3inGAN, an unconditional 3D generative model trained from 2D images of a single self-similar 3D scene. Such a model can be used to produce 3D "remixes" of a given scene, by mapping spatial latent codes into a 3D volumetric representation, which can subsequently be rendered from arbitrary views using physically based volume rendering. By construction, the generated scenes remain view-consistent across arbitrary camera configurations, without any flickering or spatio-temporal artifacts. During training, we employ a combination of 2D, obtained through differentiable volume tracing, and 3D Generative Adversarial Network (GAN) losses, across multiple scales, enforcing realism on both its 3D structure and the 2D renderings. We show results on semi-stochastic scenes of varying scale and complexity, obtained from real and synthetic sources. We demonstrate, for the first time, the feasibility of learning plausible view-consistent 3D scene variations from a single exemplar scene and provide qualitative and quantitative comparisons against recent related methods.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 58 of its 69 references that are in the map. First posted on arXiv 27 Nov 2022.

Builds on

58 of 69 references.

References · 58

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  3. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  4. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  5. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  6. 2016Density estimation using Real NVPcited by 2,029
  7. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  8. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  9. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  10. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  11. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  12. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
38 more
  1. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  2. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  3. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  4. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  5. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  6. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  7. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  8. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  9. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  10. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  11. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
  12. 2017Wasserstein GANcited by 635
  13. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  14. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  15. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  16. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
  17. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  18. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 465
  19. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  20. 2020InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANscited by 393
  21. 2019SinGAN: Learning a Generative Model from a Single Natural Imagecited by 348
  22. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 348
  23. 2019Neural Point-Based Graphicscited by 326
  24. 2020Neural Scene Graphs for Dynamic Scenescited by 298
  25. 2020State of the Art on Neural Renderingcited by 276
  26. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 267
  27. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 206
  28. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  29. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  30. 2020Adversarial Latent Autoencoderscited by 155
  31. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  32. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  33. 2019Meta-Sim: Learning to Generate Synthetic Datasetscited by 128
  34. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  35. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  36. 2020Adversarial score matching and improved sampling for image generationcited by 95
  37. 2019Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generationcited by 89
  38. 2022ReLU Fields: The Little Non-linearity That Couldcited by 87

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 5; 2024: 1; 2025: 2.

9 papers.

  1. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Imagescited by 106
  2. 2023Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shapecited by 29
  3. 2025NuiScene: Exploring Efficient Generation of Unbounded Outdoor Scenescited by 19
  4. 2023Patch-based 3D Natural Scene Generation from a Single Examplecited by 15
  5. 2022Seeing a Rose in Five Thousand Wayscited by 13
  6. 2023ShaDDR: Interactive Example-Based Geometry and Texture Generation via 3D Shape Detailization and Differentiable Renderingcited by 13
  7. 2024DECOLLAGE: 3D Detailization by Controllable, Localized, and Learned Geometry Enhancementcited by 7
  8. 2025ShapeShifter: 3D Variations Using Multiscale and Sparse Point-Voxel Diffusioncited by 3
  9. 2023GOEmbed: Gradient Origin Embeddings for Representation Agnostic 3D Feature Learningcited by 1