mlsanityData to 7 Oct 2026

Unconstrained Scene Generation with Locally Conditioned Radiance Fields

arXiv 2104.00670 ↗First posted 1 Apr 2021cs.CV cs.LGcited by 130

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We tackle the challenge of learning a distribution over complex, realistic, indoor scenes. In this paper, we introduce Generative Scene Networks (GSN), which learns to decompose scenes into a collection of many local radiance fields that can be rendered from a free moving camera. Our model can be used as a prior to generate new scenes, or to complete a scene given only sparse 2D observations. Recent work has shown that generative models of radiance fields can capture properties such as multi-view consistency and view-dependent lighting. However, these models are specialized for constrained viewing of single objects, such as cars or faces. Due to the size and complexity of realistic indoor environments, existing models lack the representational capacity to adequately capture them. Our decomposition scheme scales to larger and more complex scenes while preserving details and diversity, and the learned prior enables high-quality rendering from viewpoints that are significantly different from observed viewpoints. When compared to existing models, GSN produces quantitatively higher-quality scene renderings across several different scene datasets.

Key facts. Cited by 130 papers in the map (data through 7 Oct 2026). Builds on 48 of its 63 references that are in the map. First posted on arXiv 1 Apr 2021.

Builds on

48 of 63 references.

References · 48

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  5. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  7. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  8. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  9. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  10. 2018World Modelscited by 1,444
  11. 2015Spatial Transformer Networkscited by 1,423
  12. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
36 more
  1. 2020Neural Sparse Voxel Fieldscited by 959
  2. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  3. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  4. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  5. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  6. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  7. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  8. 2018Which Training Methods for GANs do actually Converge?cited by 672
  9. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  10. 2019Making Convolutional Networks Shift-Invariant Againcited by 390
  11. 2021GAN Inversion: A Surveycited by 366
  12. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 361
  13. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  14. 2020Free View Synthesiscited by 231
  15. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 206
  16. 2020Atlas: End-to-End 3D Scene Reconstruction from Posed Imagescited by 206
  17. 2017Learning a Multi-View Stereo Machinecited by 194
  18. 2020Neural Topological SLAM for Visual Navigationcited by 193
  19. 2019Pushing the Boundaries of View Extrapolation with Multiplane Imagescited by 191
  20. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 178
  21. 2016View Synthesis by Appearance Flowcited by 172
  22. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  23. 2020Stable View Synthesiscited by 165
  24. 2020DeRF: Decomposed Radiance Fieldscited by 163
  25. 2021Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesiscited by 140
  26. 2020Image Generators with Conditionally-Independent Pixel Synthesiscited by 105
  27. 2018Extreme View Synthesiscited by 94
  28. 2021ShaRF: Shape-conditioned Radiance Fields from a Single Viewcited by 92
  29. 2020Learning to Simulate Dynamic Environments with GameGANcited by 86
  30. 2019Neural Point Cloud Rendering via Multi-Plane Projectioncited by 60
  31. 2015Multi-view 3D Models from Single Images with a Convolutional Networkcited by 57
  32. 2017A Dataset for Developing and Benchmarking Active Visioncited by 50
  33. 2020Equivariant Neural Renderingcited by 47
  34. 2019Transformable Bottleneck Networkscited by 23
  35. 2018Generative Temporal Models with Spatial Memory for Partially Observed Environmentscited by 10
  36. 2018Incremental Scene Synthesiscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 38; 2023: 46; 2024: 21; 2025: 8; 2026: 3.

130 papers.

Sort:

By area:

  1. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  2. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  3. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  4. 2021Neural Fields in Visual Computing and Beyondcited by 539
  5. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  6. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  7. 20223D Neural Field Generation using Triplane Diffusioncited by 274
  8. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  9. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  10. 2023MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenescited by 215
  11. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusioncited by 204
  12. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
38 more
  1. 2021GeoNeRF: Generalizing NeRF with Geometry Priorscited by 195
  2. 2022From data to functa: Your data point is a function and you can treat it like onecited by 183
  3. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstructioncited by 180
  4. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  5. 2023WonderJourney: Going from Anywhere to Everywherecited by 157
  6. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  7. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  8. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  9. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  10. 2021NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenescited by 119
  11. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervisioncited by 112
  12. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Imagescited by 106
  13. 2022VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolutioncited by 101
  14. 2022Generative Neural Articulated Radiance Fieldscited by 101
  15. 2023Grid-guided Neural Radiance Fields for Large Urban Scenescited by 99
  16. 2023PanoHead: Geometry-Aware 3D Full-Head Synthesis in 360circcited by 93
  17. 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
  18. 2022Transformers as Meta-Learners for Implicit Neural Representationscited by 79
  19. 2023StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fieldscited by 79
  20. 2021Learning to Stylize Novel Viewscited by 76
  21. 2023NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Modelscited by 76
  22. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  23. 2023CityDreamer: Compositional Generative Model of Unbounded 3D Citiescited by 75
  24. 2022gDNA: Towards Generative Detailed Neural Avatarscited by 72
  25. 2023SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collectionscited by 72
  26. 2024MagicDrive3D: Controllable 3D Generation for Any-View Rendering in Street Scenescited by 70
  27. 2023InfiniCity: Infinite-Scale City Synthesiscited by 68
  28. 2022Unified Implicit Neural Stylizationcited by 67
  29. 2022Removing Objects From Neural Radiance Fieldscited by 66
  30. 2022Learning Neural Acoustic Fieldscited by 65
  31. 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 63
  32. 2023NeRFiller: Completing Scenes via Generative 3D Inpaintingcited by 62
  33. 2021InfinityGAN: Towards Infinite-Pixel Image Synthesiscited by 61
  34. 2022Implicit Neural Representations for Variable Length Human Motion Generationcited by 61
  35. 2022DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesiscited by 61
  36. 2023CC3D: Layout-Conditioned Generation of Compositional 3D Scenescited by 58
  37. 2023Learning Locally Editable Virtual Humanscited by 54
  38. 2023SceneWiz3D: Towards Text-guided 3D Scene Compositioncited by 54

50 most cited of 130 All 130 (JSON)