mlsanityData to 7 Oct 2026

HOG-Layout: Hierarchical 3D Scene Generation, Optimization and Editing via Vision-Language Models

arXiv 2604.10772 ↗First posted 12 Apr 2026cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

3D layout generation and editing play a crucial role in Embodied AI and immersive VR interaction. However, manual creation requires tedious labor, while data-driven generation often lacks diversity. The emergence of large models introduces new possibilities for 3D scene synthesis. We present HOG-Layout that enables text-driven hierarchical scene generation, optimization and real-time scene editing with large language models (LLMs) and vision-language models (VLMs). HOG-Layout improves scene semantic consistency and plausibility through retrieval-augmented generation (RAG) technology, incorporates an optimization module to enhance physical consistency, and adopts a hierarchical representation to enhance inference and optimization, achieving real-time editing. Experimental results demonstrate that HOG-Layout produces more reasonable environments compared with existing baselines, while supporting fast and intuitive scene editing.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 46 of its 103 references that are in the map. First posted on arXiv 12 Apr 2026.

Builds on

46 of 103 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Global-Local Tree Search in VLMs for 3D Indoor Scene Generation24 Mar
  2. 2024LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models3 Dec
  3. 2023GPT4Point: A Unified Framework for Point-Language Understanding and Generation5 Dec
  4. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning30 Nov

Other references · 42

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  6. 2017Billion-scale similarity search with GPUscited by 2,477
  7. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  8. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  9. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  10. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  11. 2025OpenAI GPT-5 System Cardcited by 910
  12. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
30 more
  1. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  2. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
  3. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 306
  4. 2024Long-CLIP: Unlocking the Long-Text Capability of CLIPcited by 298
  5. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
  6. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
  7. 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 170
  8. 2023ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Promptscited by 160
  9. 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesiscited by 159
  10. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  11. 2024InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Priorcited by 110
  12. 2018Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Modelscited by 105
  13. 2024GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splattingcited by 104
  14. 2023Compositional 3D Scene Generation using Locally Conditioned Diffusioncited by 101
  15. 2024I-Design: Personalized LLM Interior Designercited by 87
  16. 2023Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraintscited by 59
  17. 2023CC3D: Layout-Conditioned Generation of Compositional 3D Scenescited by 58
  18. 2024Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Modelscited by 56
  19. 2019SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentationcited by 55
  20. 2023ControlRoom3D: Room Generation using Semantic Proxy Roomscited by 54
  21. 2023RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texturecited by 53
  22. 2020End-to-End Optimization of Scene Layoutcited by 52
  23. 2020Generative Layout Modeling using Constraint Graphscited by 46
  24. 2015Text to 3D Scene Generation with Rich Lexical Groundingcited by 36
  25. 20243D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucinationcited by 36
  26. 2024Disentangled 3D Scene Generation with Layout Learningcited by 33
  27. 2025SceneEval: Evaluating Semantic Coherence in Text-Conditioned 3D Indoor Scene Synthesiscited by 26
  28. 2025FlairGPT: Repurposing LLMs for Interior Designscited by 22
  29. 2017SceneSeer: 3D Scene Design with Natural Languagecited by 20
  30. 2022COFS: Controllable Furniture layout Synthesiscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Function2Scene: 3D Indoor Scene Layout from Functional Specificationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generation
  2. 2025SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agent
  3. 2025ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editing
  4. 2026SceneMosaic: Efficient and Diverse Simulation-Ready Scene Generation via Hybrid Agentic Layout Evolution
  5. 2025Direct Numerical Layout Generation for 3D Indoor Scene Synthesis via Spatial Reasoning
  6. 2025TRELLISWorld: Training-Free World Generation from Object Generators
  7. 2025HOLODECK 2.0: Vision-Language-Guided 3D World Generation with Editing
  8. 2026HDSL: A Hierarchical Domain-Specific Language for Structured 3D Indoor Scene Generation and Localized Editing with LLM Agents