Constructing a 3D Scene from a Single Image
Not on the map. Why?
Abstract (arXiv)
Acquiring detailed 3D scenes typically demands costly equipment, multi-view data, or labor-intensive modeling. Therefore, a lightweight alternative, generating complex 3D scenes from a single top-down image, plays an essential role in real-world applications. While recent 3D generative models have achieved remarkable results at the object level, their extension to full-scene generation often leads to inconsistent geometry, layout hallucinations, and low-quality meshes. In this work, we introduce SceneFuse-3D, a training-free framework designed to synthesize coherent 3D scenes from a single top-down view. Our method is grounded in two principles: region-based generation to improve image-to-3D alignment and resolution, and spatial-aware 3D inpainting to ensure global scene coherence and high-quality geometry generation. Specifically, we decompose the input image into overlapping regions and generate each using a pretrained 3D object generator, followed by a masked rectified flow inpainting process that fills in missing geometry while maintaining structural continuity. This modular design allows us to overcome resolution bottlenecks and preserve spatial structure without requiring 3D supervision or fine-tuning. Extensive experiments across diverse scenes show that SceneFuse-3D outperforms state-of-the-art baselines, including Trellis, Hunyuan3D-2, TripoSG, and LGM, in terms of geometry quality, spatial coherence, and texture fidelity. Our results demonstrate that high-quality coherent 3D scene-level asset generation is achievable from a single top-down image using a principled, training-free pipeline.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 65 of its 67 references that are in the map. First posted on arXiv 21 May 2025.
Builds on
65 of 67 references.
References · 65
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2024GPT-4o System Cardcited by 4,588
- 2021Zero-Shot Text-to-Image Generationcited by 4,288
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
- 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
38 more
- 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
- 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
- 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
- 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
- 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
- 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 455
- 2023Emergent Correspondence from Image Diffusioncited by 446
- 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
- 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
- 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
- 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
- 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
- 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
- 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 231
- 2025TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Modelscited by 213
- 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 202
- 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 189
- 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 183
- 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesiscited by 166
- 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
- 2023WonderJourney: Going from Anywhere to Everywherecited by 158
- 2024SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Codecited by 134
- 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fieldscited by 123
- 2024CraftsMan3D: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refinercited by 123
- 2021Pathdreamer: A World Model for Indoor Navigationcited by 118
- 2024InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Priorcited by 117
- 2024GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splattingcited by 109
- 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusioncited by 109
- 20233D-GPT: Procedural 3D Modeling with Large Language Modelscited by 95
- 2024I-Design: Personalized LLM Interior Designercited by 87
- 2024BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolationcited by 83
- 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
- 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusioncited by 67
- 2024FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusioncited by 57
- 2023ControlRoom3D: Room Generation using Semantic Proxy Roomscited by 56
- 2024LT3SD: Latent Trees for 3D Scene Diffusioncited by 55
- 2023LDM3D: Latent Diffusion Model for 3Dcited by 53
50 most cited of 65 All 65 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2025G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior
- 2025Self-Evolving 3D Scene Generation from a Single Image
- 2025GEN3D: Generating Domain-Free 3D Scenes from a Single Image
- 2026Any 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale
- 2023NeRFiller: Completing Scenes via Generative 3D Inpainting
- 2025WorldGrow: Generating Infinite 3D World
- 2023Inpaint3D: 3D Scene Content Generation using 2D Inpainting Diffusion
- 2026Native3D: End-to-End 3D Scene Generation via Unified Mesh-Texture Modeling and Semantic Alignment