mlsanityData to 14 Oct 2026

Patch-based 3D Natural Scene Generation from a Single Example

arXiv 2304.12670 ↗First posted 25 Apr 2023cs.GR cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

We target a 3D generative model for general natural scenes that are typically unique and intricate. Lacking the necessary volumes of training data, along with the difficulties of having ad hoc designs in presence of varying scene characteristics, renders existing setups intractable. Inspired by classical patch-based image models, we advocate for synthesizing 3D scenes at the patch level, given a single example. At the core of this work lies important algorithmic designs w.r.t the scene representation and generative patch nearest-neighbor module, that address unique challenges arising from lifting classical 2D patch-based framework to 3D generation. These design choices, on a collective level, contribute to a robust, effective, and efficient model that can generate high-quality general natural scenes with both realistic geometric structure and visual appearance, in large quantities and varieties, as demonstrated upon a variety of exemplar scenes.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 45 of its 110 references that are in the map. First posted on arXiv 25 Apr 2023.

Builds on

45 of 110 references.

References · 45

  1. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  6. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  7. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,613
  8. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  9. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  10. 2021Volume Rendering of Neural Implicit Surfacescited by 996
  11. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 964
  12. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 905
33 more
  1. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 861
  2. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  3. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  4. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  5. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 637
  6. 2021Score-based Generative Modeling in Latent Spacecited by 636
  7. 2021Neural Fields in Visual Computing and Beyondcited by 540
  8. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  9. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 468
  10. 2019SinGAN: Learning a Generative Model from a Single Natural Imagecited by 348
  11. 2020State of the Art on Neural Renderingcited by 278
  12. 2020Free View Synthesiscited by 231
  13. 2020Stable View Synthesiscited by 165
  14. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  15. 2021GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worldscited by 98
  16. 2020Improved Techniques for Training Single-Image GANscited by 78
  17. 2020Positional Encoding as Spatial Inductive Bias in GANscited by 69
  18. 2018Deep Generative Modeling for Scene Synthesis via Hybrid Representationscited by 63
  19. 2020Multimodal Shape Completion via Conditional Generative Adversarial Networkscited by 61
  20. 2021Learning Skeletal Articulations with Neural Blend Shapescited by 53
  21. 2021Drop the GAN: In Defense of Patches Nearest Neighbors as Single Image Generative Modelscited by 51
  22. 2018GRAINS: Generative Recursive Autoencoders for INdoor Scenescited by 38
  23. 2022Learning to Generate 3D Shapes from a Single Examplecited by 31
  24. 2022Generating natural images with direct Patch Distributions Matchingcited by 27
  25. 2018SCORES: Shape Composition with Recursive Substructure Priorscited by 24
  26. 2018PhotoShape: Photorealistic Materials for Large-Scale Shape Collectionscited by 21
  27. 2021Diverse Generation from a Single Video Made Possiblecited by 13
  28. 20223inGAN: Learning a 3D Generative Model from Images of a Self-similar Scenecited by 9
  29. 2020TM-NET: Deep Generative Networks for Textured Meshescited by 8
  30. 2022SinGRAV: Learning a Generative Radiance Volume from a Single Natural Scenecited by 8
  31. 2018FrankenGAN: Guided Detail Synthesis for Building Mass-Models Using Style-Synchonized GANscited by 5
  32. 2020SketchPatch: Sketch Stylization via Seamless Patch-level Synthesiscited by 5
  33. 2017A Data-driven Approach for Furniture and Indoor Scene Colorizationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 4; 2025: 5; 2026: 2.

15 papers.

Sort:

By area:

  1. 2023Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shapecited by 29
  2. 2025NuiScene: Exploring Efficient Generation of Unbounded Outdoor Scenescited by 19
  3. 2024ThemeStation: Generating Theme-Aware 3D Assets from Few Exemplarscited by 14
  4. 2023ShaDDR: Interactive Example-Based Geometry and Texture Generation via 3D Shape Detailization and Differentiable Renderingcited by 13
  5. 2024DECOLLAGE: 3D Detailization by Controllable, Localized, and Learned Geometry Enhancementcited by 7
  6. 2026PEGAsus: 3D Personalization of Geometry and Appearancecited by 5
  7. 2025GaussianSeal: Rooting Adaptive Watermarks for 3D Gaussian Generation Modelcited by 4
  8. 2024Towards Realistic Example-based Modeling via 3D Gaussian Stitchingcited by 3
  9. 2025ShapeShifter: 3D Variations Using Multiscale and Sparse Point-Voxel Diffusioncited by 3
  10. 2025Category-Aware 3D Object Composition with Disentangled Texture and Shape Multi-view Diffusioncited by 2
  11. 2023NeRF synthesis with shading guidancecited by 1
  12. 2023GOEmbed: Gradient Origin Embeddings for Representation Agnostic 3D Feature Learningcited by 1
3 more
  1. 2026RoamScene3D: Immersive Text-to-3D Scene Generation via Adaptive Object-aware Roamingcited by 1
  2. 2024ExCellGen: Fast, Controllable, Photorealistic 3D Scene Generation from a Single Real-World Exemplarcited by 0
  3. 20253D Shape Generation: A Surveycited by 0