ExCellGen: Fast, Controllable, Photorealistic 3D Scene Generation from a Single Real-World Exemplar
Not on the map. Why?
Abstract (arXiv)
Photorealistic 3D scene generation is challenging due to the scarcity of large-scale, high-quality real-world 3D datasets and complex workflows requiring specialized expertise for manual modeling. These constraints often result in slow iteration cycles, where each modification demands substantial effort, ultimately stifling creativity. We propose a fast, exemplar-driven framework for generating 3D scenes from a single casual input, such as handheld video or drone footage. Our method first leverages 3D Gaussian Splatting (3DGS) to robustly reconstruct input scenes with a high-quality 3D appearance model. We then train a per-scene Generative Cellular Automaton (GCA) to produce a sparse volume of featurized voxels, effectively amortizing scene generation while enabling controllability. A subsequent patch-based remapping step composites the complete scene from the exemplar's initial 3D Gaussian splats, successfully recovering the appearance statistics of the input scene. The entire pipeline can be trained in less than 10 minutes for each exemplar and generates scenes in 0.5-2 seconds. Our method enables interactive creation with full user control, and we showcase complex 3D generation results from real-world exemplars within a self-contained interactive GUI.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 59 of its 77 references that are in the map. First posted on arXiv 20 Dec 2024.
Builds on
59 of 77 references.
References · 59
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
- 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
- 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
- 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
- 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,613
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
38 more
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
- 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
- 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
- 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
- 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 612
- 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
- 2023LERF: Language Embedded Radiance Fieldscited by 543
- 2023Compact 3D Gaussian Representation for Radiance Fieldcited by 372
- 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 371
- 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 364
- 2019SinGAN: Learning a Generative Model from a Single Natural Imagecited by 348
- 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 231
- 2022NeRF-Editing: Geometry Editing of Neural Radiance Fieldscited by 217
- 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 202
- 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 196
- 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 189
- 2024Reducing the Memory Footprint of 3D Gaussian Splattingcited by 133
- 2023Compositional 3D Scene Generation using Locally Conditioned Diffusioncited by 101
- 2024BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolationcited by 83
- 2023CityDreamer: Compositional Generative Model of Unbounded 3D Citiescited by 76
- 2024SCube: Instant Large-Scale Scene Reconstruction using VoxSplatscited by 60
- 2022SinFusion: Training Diffusion Models on a Single Image or Videocited by 58
- 2023CC3D: Layout-Conditioned Generation of Compositional 3D Scenescited by 58
- 2023ControlRoom3D: Room Generation using Semantic Proxy Roomscited by 56
- 2023Control3D: Towards Controllable Text-to-3D Generationcited by 46
- 2022SinDiffusion: Learning a Diffusion Model from a Single Natural Imagecited by 42
- 2024Score Distillation via Reparametrized DDIMcited by 41
- 2024LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesiscited by 39
- 2024A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesiscited by 37
- 2024Disentangled 3D Scene Generation with Layout Learningcited by 33
- 2022Learning to Generate 3D Shapes from a Single Examplecited by 31
- 2023Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shapecited by 29
- 2024Coin3D: Controllable and Interactive 3D Assets Generation with Proxy-Guided Conditioningcited by 26
- 2024SketchDream: Sketch-based Text-to-3D Generation and Editingcited by 25
- 2024SceneFactor: Factored Latent 3D Diffusion for Controllable 3D Scene Generationcited by 24
- 2021Learning to Generate 3D Shapes with Generative Cellular Automatacited by 22
- 2023CityGen: Infinite and Controllable City Layout Generationcited by 22
50 most cited of 59 All 59 (JSON)
Built on by
Not cited yet.