Scene Generation at Absolute Scale: Utilizing Semantic and Geometric Guidance From Text for Accurate and Interpretable 3D Indoor Scene Generation
On the map: Diffusion models → 3D, point cloud and scene generation
Abstract (arXiv)
We present GuidedSceneGen, a text-to-3D generation framework that produces metrically accurate, globally consistent, and semantically interpretable indoor scenes. Unlike prior text-driven methods that often suffer from geometric drift or scale ambiguity, our approach maintains an absolute world coordinate frame throughout the entire generation process. Starting from a textual scene description, we predict a global 3D layout encoding both semantic and geometric structure, which serves as a guiding proxy for downstream stages. A semantics- and depth-conditioned panoramic diffusion model then synthesizes 360deg imagery aligned with the global layout, substantially improving spatial coherence. To explore unobserved regions, we employ a video diffusion model guided by optimized camera trajectories that balances coverage and collision avoidance, achieving up to 10x faster sampling compared to exhaustive path exploration. The generated views are fused using 3D Gaussian Splatting, yielding a consistent and fully navigable 3D scene in absolute scale. GuidedSceneGen enables accurate transfer of object poses and semantic labels from layout to reconstruction, and supports progressive scene expansion without re-alignment. Quantitative results and a user study demonstrate greater 3D consistency and layout plausibility compared to recent panoramic text-to-3D baselines.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 59 of its 64 references that are in the map. First posted on arXiv 14 Mar 2026.
Builds on
59 of 64 references.
Closest ancestors · 12
Same direction, newest first
- 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generation4 Jun
- 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Models18 Mar
- 2025Bolt3D: Generating 3D Scenes in Seconds18 Mar
- 2025CubeDiff: Repurposing Diffusion-Based Image Models for Panorama Generation28 Jan
- 2024Wonderland: Navigating 3D Scenes from a Single Image16 Dec
- 2024VistaDream: Sampling multiview consistent images for single-view scene reconstruction22 Oct
- 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis3 Sep
- 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusion10 Apr
- 2023Diffusion360: Seamless 360 Degree Panoramic Image Generation based on Diffusion Models22 Nov
- 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
- 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields19 May
- 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
Other references · 47
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2016Improved Techniques for Training GANscited by 2,938
- 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
35 more
- 2022Video Diffusion Modelscited by 2,280
- 2024Depth Anything V2cited by 1,385
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
- 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 295
- 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 285
- 2024PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstructioncited by 280
- 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
- 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
- 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 210
- 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
- 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
- 2017Learning to Predict Indoor Illumination from a Single Imagecited by 160
- 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
- 2023WonderJourney: Going from Anywhere to Everywherecited by 157
- 2025MEt3R: Measuring Multi-View Consistency in Generated Imagescited by 91
- 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
- 2024DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splattingcited by 81
- 2023Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenescited by 75
- 2024DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Samplingcited by 72
- 2023Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraintscited by 59
- 2024ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidancecited by 59
- 2023ControlRoom3D: Room Generation using Semantic Proxy Roomscited by 54
- 2023SceneWiz3D: Towards Text-guided 3D Scene Compositioncited by 54
- 2024LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generationcited by 53
- 2024SceneCraft: Layout-Guided 3D Scene Generationcited by 40
- 2025FlexWorld: Progressively Expanding 3D Scenes for Flexiable-View Synthesiscited by 37
- 2024Disentangled 3D Scene Generation with Layout Learningcited by 33
- 2025WorldExplorer: Towards Generating Fully Navigable 3D Scenescited by 26
- 2025SPATIALGEN: Layout-guided 3D Indoor Scene Generationcited by 14
- 2025Generating 360deg Video is What You Need For a 3D Scenecited by 5
- 2025DreamAnywhere: Object-Centric Panoramic 3D Scene Generationcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Layout2Scene: 3D Semantic Layout Guided Scene Generation via Geometry and Appearance Diffusion Priors
- 2026SDesc3D: Towards Layout-Aware 3D Indoor Scene Generation from Short Descriptions
- 2025ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editing
- 2025M3DLayout: A Multi-Source Dataset of 3D Indoor Layouts and Structured Descriptions for 3D Generation
- 2026Global Graph-Validated Optimization for VLM-based 3D Indoor Scene Generation
- 2025GeoSceneGraph: Geometric Scene Graph Diffusion Model for Text-guided 3D Indoor Scene Synthesis
- 2025SemLayoutDiff: Semantic Layout Generation with Diffusion Model for Indoor Scene Synthesis
- 2025MVRoom: Controllable 3D Indoor Scene Generation with Multi-View Diffusion Models