mlsanityData to 7 Oct 2026

GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs

arXiv 2312.00093 ↗First posted 30 Nov 2023cs.CV cs.GR cs.LGcited by 96

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

As pretrained text-to-image diffusion models become increasingly powerful, recent efforts have been made to distill knowledge from these text-to-image pretrained models for optimizing a text-guided 3D model. Most of the existing methods generate a holistic 3D model from a plain text input. This can be problematic when the text describes a complex scene with multiple objects, because the vectorized text embeddings are inherently unable to capture a complex description with multiple entities and relationships. Holistic 3D modeling of the entire scene further prevents accurate grounding of text entities and concepts. To address this limitation, we propose GraphDreamer, a novel framework to generate compositional 3D scenes from scene graphs, where objects are represented as nodes and their interactions as edges. By exploiting node and edge information in scene graphs, our method makes better use of the pretrained text-to-image diffusion model and is able to fully disentangle different objects without image-level supervision. To facilitate modeling of object-wise relationships, we use signed distance fields as representation and impose a constraint to avoid inter-penetration of objects. To avoid manual scene graph creation, we design a text prompt for ChatGPT to generate scene graphs based on text inputs. We conduct both qualitative and quantitative experiments to validate the effectiveness of GraphDreamer in generating high-fidelity compositional 3D scenes with disentangled object entities.

Key facts. Cited by 96 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 37 of its 46 references that are in the map. First posted on arXiv 30 Nov 2023.

Builds on

37 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Divide & Bind Your Attention for Improved Generative Semantic Nursing20 Jul
  2. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May

Other references · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Classifier-Free Diffusion Guidancecited by 5,717
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  6. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  8. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  9. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  10. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  11. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  12. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
22 more
  1. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  2. 2021Variational Diffusion Modelscited by 1,145
  3. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  4. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  5. 2020Implicit Geometric Regularization for Learning Shapescited by 707
  6. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  7. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  8. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  9. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  10. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  11. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  12. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  13. 2021Towards Causal Representation Learningcited by 207
  14. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Modelscited by 151
  15. 2023Compositional 3D Scene Generation using Locally Conditioned Diffusioncited by 101
  16. 2021Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphscited by 83
  17. 2022Capturing and Animation of Body and Clothing from Monocular Videocited by 83
  18. 2023CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layoutcited by 57
  19. 2023Learning Disentangled Avatars with Hybrid 3D Representationscited by 44
  20. 2023Text-Guided Generation and Editing of Compositional 3D Avatarscited by 39
  21. 2022Hybrid Mesh-neural Representation for 3D Transparent Object Reconstructioncited by 15
  22. 2022Structural Causal 3D Reconstructioncited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 34; 2025: 36; 2026: 26.

96 papers.

Sort:

By area:

  1. 2024CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generationcited by 163
  2. 2024DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splattingcited by 81
  3. 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformerscited by 78
  4. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  5. 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusioncited by 66
  6. 2024SINGAPO: Single Image Controlled Generation of Articulated Parts in Objectscited by 66
  7. 2024ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidancecited by 59
  8. 2024InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Modelscited by 53
  9. 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Passcited by 49
  10. 2024Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understandingcited by 46
  11. 2024REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignmentcited by 45
  12. 2024Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databasescited by 43
38 more
  1. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
  2. 2024SceneCraft: Layout-Guided 3D Scene Generationcited by 40
  3. 2024The Scene Language: Representing Scenes with Programs, Words, and Embeddingscited by 31
  4. 2024Mixed Diffusion for 3D Indoor Scene Synthesiscited by 30
  5. 2024Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNetscited by 30
  6. 2024AutoVFX: Physically Realistic Video Editing from Natural Language Instructionscited by 27
  7. 2025Direct Numerical Layout Generation for 3D Indoor Scene Synthesis via Spatial Reasoningcited by 27
  8. 2024Frankenstein: Generating Semantic-Compositional 3D Scenes in One Tri-Planecited by 26
  9. 20243DGraphLLM: Combining Semantic Graphs and Large Language Models for 3D Scene Understandingcited by 26
  10. 2024AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animationcited by 25
  11. 2025Embodied AI: From LLMs to World Modelscited by 23
  12. 2024Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprintcited by 21
  13. 2025WorldCraft: Photo-Realistic 3D World Creation and Customization via LLM Agentscited by 21
  14. 2024Compositional 3D-aware Video Generation with LLM Directorcited by 18
  15. 2024PuzzleAvatar: Assembling 3D Avatars from Personal Albumscited by 16
  16. 2026MV-SAM3D: Adaptive Multi-View Fusion for Layout-Aware 3D Generationcited by 16
  17. 2026Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesiscited by 15
  18. 2025Agentic 3D Scene Generation with Spatially Contextualized VLMscited by 14
  19. 2025OptiScene: LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimizationcited by 14
  20. 2025HOLODECK 2.0: Vision-Language-Guided 3D World Generation with Editingcited by 14
  21. 2025PAT3D: Physics-Augmented Text-to-3D Scene Generationcited by 14
  22. 2024Text-to-3D Shape Generationcited by 13
  23. 2025InteractAnything: Zero-shot Human Object Interaction Synthesis via LLM Feedback and Object Affordance Parsingcited by 13
  24. 2024CYCLO: Cyclic Graph Transformer Approach to Multi-Object Relationship Modeling in Aerial Videoscited by 11
  25. 2024Diffusion Models in 3D Vision: A Surveycited by 11
  26. 2024CompGS: Unleashing 2D Compositionality for Compositional Text-to-3D via Dynamically Optimizing 3D Gaussianscited by 10
  27. 2024Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modelingcited by 10
  28. 2025Layout2Scene: 3D Semantic Layout Guided Scene Generation via Geometry and Appearance Diffusion Priorscited by 10
  29. 2025FROSS: Faster-than-Real-Time Online 3D Semantic Scene Graph Generation from RGB-D Imagescited by 10
  30. 2024Planner3D: LLM-enhanced graph prior meets 3D indoor scene explicit regularizationcited by 9
  31. 2024Semantic Score Distillation Sampling for Compositional Text-to-3D Generationcited by 9
  32. 2025LLM Meets Scene Graph: Can Large Language Models Understand and Generate Scene Graphs? A Benchmark and Empirical Studycited by 9
  33. 2024DreamDissector: Learning Disentangled Text-to-3D Generation from 2D Diffusion Priorscited by 8
  34. 2024DreamScape: 3D Scene Creation via Gaussian Splatting joint Correlation Modelingcited by 7
  35. 2025Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Modelscited by 7
  36. 2024Scene123: One Prompt to 3D Scene Generation via Video-Assisted and Consistency-Enhanced MAEcited by 6
  37. 2025Is Single-View Mesh Reconstruction Ready for Robotics?cited by 6
  38. 2025Towards Geometric and Textural Consistency 3D Scene Generation via Single Image-guided Model Generation and Layout Optimizationcited by 6

50 most cited of 96 All 96 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Dependency-Aware Discrete Diffusion for Scene Graph Generation
  2. 20233D Scene Diffusion Guidance using Scene Graphs
  3. 2025GeoSceneGraph: Geometric Scene Graph Diffusion Model for Text-guided 3D Indoor Scene Synthesis
  4. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusion
  5. 2023SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesis
  6. 2024SG-Adapter: Enhancing Text-to-Image Generation with Scene Graph Guidance
  7. 2023SceneWiz3D: Towards Text-guided 3D Scene Composition
  8. 2023CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusion