mlsanityData to 7 Oct 2026

Controllable 3D Outdoor Scene Generation via Scene Graphs

arXiv 2503.07152 ↗First posted 10 Mar 2025cs.CVcited by 9

On the map: World models → Video-generation world models and simulators

also in 1 more
Abstract (arXiv)

Three-dimensional scene generation is crucial in computer vision, with applications spanning autonomous driving, gaming and the metaverse. Current methods either lack user control or rely on imprecise, non-intuitive conditions. In this work, we propose a method that uses, scene graphs, an accessible, user friendly control format to generate outdoor 3D scenes. We develop an interactive system that transforms a sparse scene graph into a dense BEV (Bird's Eye View) Embedding Map, which guides a conditional diffusion model to generate 3D scenes that match the scene graph description. During inference, users can easily create or modify scene graphs to generate large-scale outdoor scenes. We create a large-scale dataset with paired scene graphs and 3D semantic scenes to train the BEV embedding and diffusion models. Experimental results show that our approach consistently produces high-quality 3D urban scenes closely aligned with the input scene graphs. To the best of our knowledge, this is the first approach to generate 3D outdoor scenes conditioned on scene graphs.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 53 of its 70 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

53 of 70 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  2. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 51

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2017Graph Attention Networkscited by 8,854
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  5. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  6. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  7. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  8. 2016Variational Graph Auto-Encoderscited by 1,373
  9. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  10. 2022Diffusion Models in Vision: A Surveycited by 1,008
  11. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  12. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
38 more
  1. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  2. 2018Image Generation from Scene Graphscited by 324
  3. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  4. 20193D Scene Graph: A Structure for Unified Semantics, 3D Space, and Cameracited by 280
  5. 2018Auto-Encoding Scene Graphs for Image Captioningcited by 254
  6. 2023Language is All a Graph Needscited by 219
  7. 2020Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructionscited by 205
  8. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 183
  9. 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesiscited by 159
  10. 2020SceneFormer: Indoor Scene Generation with Transformerscited by 154
  11. 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
  12. 2023GraphText: Graph Reasoning in Text Spacecited by 110
  13. 2024InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Priorcited by 110
  14. 2021Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphscited by 83
  15. 2023CityDreamer: Compositional Generative Model of Unbounded 3D Citiescited by 75
  16. 2023InfiniCity: Infinite-Scale City Synthesiscited by 68
  17. 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusioncited by 66
  18. 2023Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraintscited by 59
  19. 2022Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Trainingcited by 56
  20. 2019SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentationcited by 55
  21. 2023CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusioncited by 54
  22. 2024SemCity: Semantic Scene Generation with Triplane Diffusioncited by 53
  23. 2020Generative Layout Modeling using Constraint Graphscited by 46
  24. 2023Pyramid Diffusion for Fine 3D Large Scene Generationcited by 44
  25. 2023SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesiscited by 41
  26. 2022MotionSC: Data Set and Network for Real-Time Semantic Mapping in Dynamic Environmentscited by 40
  27. 2024SceneCraft: Layout-Guided 3D Scene Generationcited by 40
  28. 2024SpaRP: Fast 3D Object Reconstruction and Pose Estimation from Sparse Viewscited by 35
  29. 2023Diffusion Probabilistic Models for Scene-Scale 3D Categorical Datacited by 33
  30. 2024Mixed Diffusion for 3D Indoor Scene Synthesiscited by 30
  31. 2020Campus3D: A Photogrammetry Point Cloud Benchmark for Hierarchical Understanding of Outdoor Scenecited by 27
  32. 2024Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular Transformercited by 26
  33. 2024SceneFactor: Factored Latent 3D Diffusion for Controllable 3D Scene Generationcited by 24
  34. 2023CityGen: Infinite and Controllable City Layout Generationcited by 22
  35. 2025MMGDreamer: Mixed-Modality Graph for Geometry-Controllable 3D Indoor Scene Generationcited by 18
  36. 2024Urban Scene Diffusion through Semantic Occupancy Mapcited by 14
  37. 2024Isotropic3D: Image-to-3D Generation Based on a Single CLIP Embeddingcited by 13
  38. 2024Planner3D: LLM-enhanced graph prior meets 3D indoor scene explicit regularizationcited by 9

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 4.

9 papers.

  1. 20253D and 4D World Modeling: A Surveycited by 64
  2. 20253D Scene Generation: A Surveycited by 24
  3. 2025LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequencescited by 14
  4. 2025HoliGS: Holistic Gaussian Splatting for Embodied View Synthesiscited by 7
  5. 2026ActionParty: Multi-Subject Action Binding in Generative Video Gamescited by 4
  6. 2025FOGMACHINE -- Leveraging Discrete-Event Simulation and Scene Graphs for Modeling Hierarchical, Interconnected Environments under Partial Observations from Mobile Agentscited by 0
  7. 2026OpenSGA: Efficient 3D Scene Graph Alignment in the Open Worldcited by 0
  8. 2026NoPA: Non-Parametric Online 3D Scene Graph Generationcited by 0
  9. 2026WorldWeave: Growing Persistent Geometric Worlds for Video Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ScenDi: 3D-to-2D Scene Diffusion Cascades for Urban Generation
  2. 2026SEM-ROVER: Semantic Voxel-Guided Diffusion for Large-Scale Driving Scene Generation
  3. 2025MeSS: City Mesh-Guided Outdoor Scene Generation with Cross-View Consistent Diffusion
  4. 2024Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion
  5. 20233D Scene Diffusion Guidance using Scene Graphs
  6. 2024MagicDrive3D: Controllable 3D Generation for Any-View Rendering in Street Scenes
  7. 2025GeoSceneGraph: Geometric Scene Graph Diffusion Model for Text-guided 3D Indoor Scene Synthesis
  8. 2026WorldFlow3D: Flowing Through 3D Distributions for Unbounded World Generation