# Graph Canvas for Controllable 3D Scene Generation

arXiv: https://arxiv.org/abs/2412.00091 · HTML: https://mlsanity.com/paper/2412.00091/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 27 Nov 2024.

## Builds on (most cited first, 23 of 23)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [Point-E: A System for Generating 3D Point Clouds from Complex Prompts](https://mlsanity.com/paper/2212.08751/) (2022)
- [LayoutGPT: Compositional Visual Planning and Generation with Large Language Models](https://mlsanity.com/paper/2305.15393/) (2023)
- [GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models](https://mlsanity.com/paper/2310.08529/) (2023)
- [ATISS: Autoregressive Transformers for Indoor Scene Synthesis](https://mlsanity.com/paper/2110.03675/) (2021)
- [SpatialBot: Precise Spatial Understanding with Vision Language Models](https://mlsanity.com/paper/2406.13642/) (2024)
- [DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generation](https://mlsanity.com/paper/2306.12422/) (2023)
- [GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modeling](https://mlsanity.com/paper/2403.19655/) (2024)
- [Delay-sensitive Task Offloading in Vehicular Fog Computing-Assisted Platoons](https://mlsanity.com/paper/2309.10234/) (2023)
- [Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprint](https://mlsanity.com/paper/2410.15391/) (2024)
- [End-to-End Evaluation for Low-Latency Simultaneous Speech Translation](https://mlsanity.com/paper/2308.03415/) (2023)
- [Smart Policy Control for Securing Federated Learning Management System](https://mlsanity.com/paper/2305.09134/) (2023)
- [A systematic review of geospatial location embedding approaches in large language models: A path to spatial AI systems](https://mlsanity.com/paper/2401.10279/) (2024)
- [Performance comparison of timing-based anomaly detectors for Controller Area Network: a reproducible study](https://mlsanity.com/paper/2307.04561/) (2023)
- [Quantum Encryption of superposition states with Quantum Permutation Pad in IBM Quantum Computers](https://mlsanity.com/paper/2301.10832/) (2023)
- [To Wake-up or Not to Wake-up: Reducing Keyword False Alarm by Successive Refinement](https://mlsanity.com/paper/2304.03416/) (2023)
- [DABS: Data-Agnostic Backdoor attack at the Server in Federated Learning](https://mlsanity.com/paper/2305.01267/) (2023)
- [Optimal Eventual Byzantine Agreement Protocols with Omission Failures](https://mlsanity.com/paper/2305.06271/) (2023)
- [Semantic Analysis of Subexponential Modalities in Distributive Non-commutative Linear Logic](https://mlsanity.com/paper/2308.04521/) (2023)
- [Spatial Computing: Concept, Applications, Challenges and Future Directions](https://mlsanity.com/paper/2402.07912/) (2024)

## Built on by (most cited first, 11 of 11)

- [ConeSep: Cone-based Robust Noise-Unlearning Compositional Network for Composed Image Retrieval](https://mlsanity.com/paper/2604.20358/) (2026)
- [ChatMotion: A Multimodal Multi-Agent for Human Motion Analysis](https://mlsanity.com/paper/2502.18180/) (2025)
- [RAM: Recover Any 3D Human Motion in-the-Wild](https://mlsanity.com/paper/2603.19929/) (2026)
- [Bayesian Optimization for Controlled Image Editing via LLMs](https://mlsanity.com/paper/2502.18116/) (2025)
- [SDesc3D: Towards Layout-Aware 3D Indoor Scene Generation from Short Descriptions](https://mlsanity.com/paper/2604.01972/) (2026)
- [Grounding Multi-Hop Reasoning in Structural Causal Models via Group Relative Policy Optimization](https://mlsanity.com/paper/2605.01482/) (2026)
- [Self-Correction Can Amplify Hallucinations: Fact-Level Repair with Graph-Based Evidence Routing in Multimodal Generation](https://mlsanity.com/paper/2606.00232/) (2026)
- [Improving Multimodal Reasoning via Worst Dimension Optimization](https://mlsanity.com/paper/2606.07801/) (2026)
- [VigilFormer: Deformable Attention for Video Anomaly Detection with Causal Risk Inference](https://mlsanity.com/paper/2606.14724/) (2026)
- [Bridging Geographic Bias in Urban Streetscape Inference via Lifelong Learning with Visual-Semantic Pivoting](https://mlsanity.com/paper/2606.15055/) (2026)
- [SymDiag: Explainable Diagnosis for LLM Reasoning via Neuro-Symbolic Verification](https://mlsanity.com/paper/2608.08786/) (2026)

## Abstract (arXiv)

Spatial intelligence is foundational to AI systems that interact with the physical world, particularly in 3D scene generation and spatial comprehension. Current methodologies for 3D scene generation often rely heavily on predefined datasets, and struggle to adapt dynamically to changing spatial relationships. In this paper, we introduce GraphCanvas3D, a programmable, extensible, and adaptable framework for controllable 3D scene generation. Leveraging in-context learning, GraphCanvas3D enables dynamic adaptability without the need for retraining, supporting flexible and customizable scene creation. Our framework employs hierarchical, graph-driven scene descriptions, representing spatial elements as graph nodes and establishing coherent relationships among objects in 3D environments. Unlike conventional approaches, which are constrained in adaptability and often require predefined input masks or retraining for modifications, GraphCanvas3D allows for seamless object manipulation and scene adjustments on the fly. Additionally, GraphCanvas3D supports 4D scene generation, incorporating temporal dynamics to model changes over time. Experimental results and user studies demonstrate that GraphCanvas3D enhances usability, flexibility, and adaptability for scene generation. Our code and models are available on the project website: https://github.com/ILGLJ/Graph-Canvas.