# SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collections

arXiv: https://arxiv.org/abs/2302.01330 · HTML: https://mlsanity.com/paper/2302.01330/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- Builds on 61 of its 83 references that are in the map.
- First posted on arXiv 2 Feb 2023.

## Builds on (most cited first, 30 of 61)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](https://mlsanity.com/paper/2103.13744/) (2021)
- [RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs](https://mlsanity.com/paper/2112.00724/) (2021)

## Built on by (most cited first, 30 of 72)

- [SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis](https://mlsanity.com/paper/2303.16196/) (2023)
- [XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchies](https://mlsanity.com/paper/2312.03806/) (2023)
- [BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolation](https://mlsanity.com/paper/2401.17053/) (2024)
- [CityDreamer: Compositional Generative Model of Unbounded 3D Cities](https://mlsanity.com/paper/2309.00610/) (2023)
- [Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenes](https://mlsanity.com/paper/2303.13450/) (2023)
- [EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusion](https://mlsanity.com/paper/2405.00915/) (2024)
- [NeRFiller: Completing Scenes via Generative 3D Inpainting](https://mlsanity.com/paper/2312.04560/) (2023)
- [CC3D: Layout-Conditioned Generation of Compositional 3D Scenes](https://mlsanity.com/paper/2303.12074/) (2023)
- [LT3SD: Latent Trees for 3D Scene Diffusion](https://mlsanity.com/paper/2409.08215/) (2024)
- [Persistent Nature: A Generative Model of Unbounded 3D Worlds](https://mlsanity.com/paper/2303.13515/) (2023)
- [Pyramid Diffusion for Fine 3D Large Scene Generation](https://mlsanity.com/paper/2311.12085/) (2023)
- [Generative Gaussian Splatting for Unbounded 3D City Generation](https://mlsanity.com/paper/2406.06526/) (2024)
- [Free4D: Tuning-free 4D Scene Generation with Spatial-Temporal Consistency](https://mlsanity.com/paper/2503.20785/) (2025)
- [PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation](https://mlsanity.com/paper/2312.04559/) (2023)
- [SynCity: Training-Free Generation of 3D Worlds](https://mlsanity.com/paper/2503.16420/) (2025)
- [3D-SceneDreamer: Text-Driven 3D-Consistent Scene Generation](https://mlsanity.com/paper/2403.09439/) (2024)
- [AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillation](https://mlsanity.com/paper/2312.03795/) (2023)
- [A Comprehensive Survey on 3D Content Generation](https://mlsanity.com/paper/2402.01166/) (2024)
- [UrbanGIRAFFE: Representing Urban Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2303.14167/) (2023)
- [CityCraft: A Real Crafter for 3D City Generation](https://mlsanity.com/paper/2406.04983/) (2024)
- [3DitScene: Editing Any Scene via Language-guided Disentangled Gaussian Splatting](https://mlsanity.com/paper/2405.18424/) (2024)
- [FirePlace: Geometric Refinements of LLM Common Sense Reasoning for 3D Object Placement](https://mlsanity.com/paper/2503.04919/) (2025)
- [UrbanWorld: An Urban World Model for 3D City Generation](https://mlsanity.com/paper/2407.11965/) (2024)
- [SceneFactor: Factored Latent 3D Diffusion for Controllable 3D Scene Generation](https://mlsanity.com/paper/2412.01801/) (2024)
- [3D Scene Generation: A Survey](https://mlsanity.com/paper/2505.05474/) (2025)
- [CityGen: Infinite and Controllable City Layout Generation](https://mlsanity.com/paper/2312.01508/) (2023)
- [CityX: Controllable Procedural Content Generation for Unbounded 3D Cities](https://mlsanity.com/paper/2407.17572/) (2024)
- [WorldCraft: Photo-Realistic 3D World Creation and Customization via LLM Agents](https://mlsanity.com/paper/2502.15601/) (2025)
- [Objaverse++: Curated 3D Object Dataset with Quality Annotations](https://mlsanity.com/paper/2504.07334/) (2025)
- [Make-It-4D: Synthesizing a Consistent Long-Term Dynamic Scene Video from a Single Image](https://mlsanity.com/paper/2308.10257/) (2023)

## Abstract (arXiv)

In this work, we present SceneDreamer, an unconditional generative model for unbounded 3D scenes, which synthesizes large-scale 3D landscapes from random noise. Our framework is learned from in-the-wild 2D image collections only, without any 3D annotations. At the core of SceneDreamer is a principled learning paradigm comprising 1) an efficient yet expressive 3D scene representation, 2) a generative scene parameterization, and 3) an effective renderer that can leverage the knowledge from 2D images. Our approach begins with an efficient bird's-eye-view (BEV) representation generated from simplex noise, which includes a height field for surface elevation and a semantic field for detailed scene semantics. This BEV scene representation enables 1) representing a 3D scene with quadratic complexity, 2) disentangled geometry and semantics, and 3) efficient training. Moreover, we propose a novel generative neural hash grid to parameterize the latent space based on 3D positions and scene semantics, aiming to encode generalizable features across various scenes. Lastly, a neural volumetric renderer, learned from 2D image collections through adversarial training, is employed to produce photorealistic images. Extensive experiments demonstrate the effectiveness of SceneDreamer and superiority over state-of-the-art methods in generating vivid yet diverse unbounded 3D worlds.