AccioScene: Compositional 3D Scene Generation via Graph Diffusion and Interaction-driven Critics
Not on the map. Why?
Abstract (arXiv)
This paper presents a framework for generating 3D indoor scenes from text prompts. Existing methods often formulate scene synthesis as an object layout prediction problem conditioned on a single input modality, such as a text description, room shape, or scene graph. This design can lead to object collisions and limited functional plausibility, reducing its practical applicability. To address these limitations, we introduce a multi-stage pipeline that better reflects practical scene creation scenarios. Given a text prompt describing partial scene content, our method first uses graph diffusion to produce a contextually coherent scene graph and then predicts a realistic object layout. In addition, we incorporate lightweight human-object interaction priors to encourage human-centric and functional arrangements, with explicit spatial constraints to reduce interpenetration. Our approach generates coherent 3D scenes with viable layouts that better support human interaction. Experiments on the 3D-FRONT dataset demonstrate that our method achieves competitive or state-of-the-art performance compared with existing approaches, while improving the physical plausibility of generated scenes.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 36 of its 46 references that are in the map. First posted on arXiv 5 Feb 2025.
Builds on
36 of 46 references.
References · 36
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
- 2023Qwen Technical Reportcited by 3,344
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
- 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
- 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
- 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
- 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 380
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
24 more
- 20203D-FUTURE: 3D Furniture shape with TextUREcited by 311
- 2021AGORA: Avatars in Geography Optimized for Regression Analysiscited by 205
- 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 184
- 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesiscited by 166
- 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
- 2020SceneFormer: Indoor Scene Generation with Transformerscited by 158
- 2024PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AIcited by 125
- 2024InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Priorcited by 117
- 2023Controllable Human-Object Interaction Synthesiscited by 109
- 2024GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splattingcited by 109
- 2018Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Modelscited by 106
- 2023GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphscited by 97
- 2024I-Design: Personalized LLM Interior Designercited by 87
- 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 82
- 2024Generating Human Interaction Motions in Scenes with Text Controlcited by 80
- 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusioncited by 67
- 2022MIME: Human-Aware 3D Scene Generationcited by 57
- 2024SceneTeller: Language-to-3D Scene Generationcited by 43
- 2021Pose2Room: Understanding 3D Scenes from Human Activitiescited by 22
- 2023Scene Synthesis from Human Motioncited by 20
- 2020Modeling Artistic Workflows for Image Generation and Editingcited by 11
- 2024VRCopilot: Authoring 3D Layouts with Generative AI Models in VRcited by 10
- 2024Planner3D: LLM-enhanced graph prior meets 3D indoor scene explicit regularizationcited by 9
- 2024Human-Aware 3D Scene Generation with Spatially-constrained Diffusion Modelscited by 5
Built on by
Citing papers per year: 2025: 1.
1 paper.