mlsanityData to 7 Oct 2026

AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homes

arXiv 2312.06644 ↗First posted 11 Dec 2023cs.CV cs.AI cs.GRcited by 81

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Inspired by cognitive theories, we introduce AnyHome, a framework that translates any text into well-structured and textured indoor scenes at a house-scale. By prompting Large Language Models (LLMs) with designed templates, our approach converts provided textual narratives into amodal structured representations. These representations guarantee consistent and realistic spatial layouts by directing the synthesis of a geometry mesh within defined constraints. A Score Distillation Sampling process is then employed to refine the geometry, followed by an egocentric inpainting process that adds lifelike textures to it. AnyHome stands out with its editability, customizability, diversity, and realism. The structured representations for scenes allow for extensive editing at varying levels of granularity. Capable of interpreting texts ranging from simple labels to detailed narratives, AnyHome generates detailed geometries and textures that outperform existing methods in both quantitative and qualitative measures.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 50 of its 97 references that are in the map. First posted on arXiv 11 Dec 2023.

Builds on

50 of 97 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  2. 2021Learning Transferable Visual Models From Natural Language Supervision26 Feb

Other references · 48

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2023A Survey of Large Language Modelscited by 2,537
  4. 2022Emergent Abilities of Large Language Modelscited by 2,483
  5. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  6. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  7. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  8. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  9. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  10. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  11. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  12. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
36 more
  1. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
  2. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  3. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
  4. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 306
  5. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 303
  6. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
  7. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  8. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  9. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  10. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Modelscited by 234
  11. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 210
  12. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  13. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
  14. 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
  15. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 162
  16. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  17. 2020SceneFormer: Indoor Scene Generation with Transformerscited by 154
  18. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  19. 2020House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generationcited by 86
  20. 2023TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Modelscited by 86
  21. 2023LEGO-Net: Learning Regular Rearrangements of Objects in Roomscited by 75
  22. 2018Human-centric Indoor Scene Synthesis Using Stochastic Grammarcited by 73
  23. 2022HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoisingcited by 72
  24. 2022Towards Implicit Text-Guided 3D Shape Generationcited by 68
  25. 2018Deep Generative Modeling for Scene Synthesis via Hybrid Representationscited by 63
  26. 2022ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Modelcited by 62
  27. 2023CC3D: Layout-Conditioned Generation of Compositional 3D Scenescited by 58
  28. 2023RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texturecited by 53
  29. 2022CLIP-Sculptor: Zero-Shot Generation of High-Fidelity and Diverse Shapes from Natural Languagecited by 46
  30. 2023X-Mesh: Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidancecited by 37
  31. 2023TAPS3D: Text-Guided 3D Textured Shape Generation from Pseudo Supervisioncited by 28
  32. 2021Indoor Scene Generation from a Collection of Semantic-Segmented Depth Imagescited by 27
  33. 2017SceneSeer: 3D Scene Design with Natural Languagecited by 20
  34. 2023Graph Transformer GANs for Graph-Constrained House Generationcited by 19
  35. 2023Text2Scene: Text-driven Indoor Scene Stylization with Part-aware Detailscited by 18
  36. 2023Aladdin: Zero-Shot Hallucination of Stylized 3D Assets from Abstract Scene Descriptionscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 32; 2026: 33.

81 papers.

Sort:

By area:

  1. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  2. 2024GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splattingcited by 104
  3. 2024LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Modelscited by 94
  4. 2024I-Design: Personalized LLM Interior Designercited by 87
  5. 2025SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agentcited by 53
  6. 2024DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Modelscited by 50
  7. 2026SAGE: Scalable Agentic 3D Scene Generation for Embodied AIcited by 40
  8. 2024Architect: Generating Vivid and Interactive 3D Scenes with Hierarchical 2D Inpaintingcited by 37
  9. 2024Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Modelscited by 36
  10. 2025Global-Local Tree Search in VLMs for 3D Indoor Scene Generationcited by 29
  11. 2025BlenderGym: Benchmarking Foundational Model Systems for Graphics Editingcited by 27
  12. 2025ArtiScene: Language-Driven Artistic 3D Scene Generation Through Image Intermediarycited by 27
38 more
  1. 2025Direct Numerical Layout Generation for 3D Indoor Scene Synthesis via Spatial Reasoningcited by 27
  2. 2024Frankenstein: Generating Semantic-Compositional 3D Scenes in One Tri-Planecited by 26
  3. 2025SceneEval: Evaluating Semantic Coherence in Text-Conditioned 3D Indoor Scene Synthesiscited by 26
  4. 2025FirePlace: Geometric Refinements of LLM Common Sense Reasoning for 3D Object Placementcited by 24
  5. 20253D Scene Generation: A Surveycited by 24
  6. 2025HSM: Hierarchical Scene Motifs for Multi-Scale Indoor Scene Generationcited by 22
  7. 2024Chat2Layout: Interactive 3D Furniture Layout with a Multimodal LLMcited by 20
  8. 2024DeBaRA: Denoising-Based 3D Room Arrangement Generationcited by 20
  9. 2024BlenderAlchemy: Editing 3D Graphics with Vision-Language Modelscited by 18
  10. 2025Agentic 3D Scene Generation with Spatially Contextualized VLMscited by 14
  11. 2025HOLODECK 2.0: Vision-Language-Guided 3D World Generation with Editingcited by 14
  12. 2025FloorplanQA: A Benchmark for Spatial Reasoning in LLMs using Structured Representationscited by 13
  13. 2024LEGENT: Open Platform for Embodied Agentscited by 12
  14. 2024Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modelingcited by 10
  15. 2025BIMgent: Towards Autonomous Building Modeling via Computer-use Agentscited by 10
  16. 2024HouseCrafter: Lifting Floorplans to 3D Scenes with 2D Diffusion Modelcited by 9
  17. 2025RoomCraft: Controllable and Complete 3D Indoor Scene Generationcited by 9
  18. 2025PlaceIt3D: Language-Guided Object Placement in Real 3D Scenescited by 8
  19. 2025FreeScene: Mixed Graph Diffusion for 3D Scene Synthesis from Free Promptscited by 8
  20. 2026MANSION: Multi-floor lANguage-to-3D Scene generatIOn for loNg-horizon taskscited by 8
  21. 2025Video Perception Models for 3D Scene Synthesiscited by 7
  22. 2026PhyScensis: Physics-Augmented LLM Agents for Complex Physical Scene Arrangementcited by 7
  23. 2026SceneCode: Executable World Programs for Editable Indoor Scenes with Articulated Objectscited by 7
  24. 2025SemLayoutDiff: Semantic Layout Generation with Diffusion Model for Indoor Scene Synthesiscited by 6
  25. 2025From Programs to Poses: Factored Real-World Scene Generation via Learned Program Librariescited by 6
  26. 2024DStruct2Design: Data and Benchmarks for Data Structure Driven Generative Floor Plan Designcited by 5
  27. 2026AnchoredDream: Zero-Shot 360deg Indoor Scene Generation from a Single View via Geometric Groundingcited by 5
  28. 2026SceneAssistant: A Visual Feedback Agent for Open-Vocabulary 3D Scene Generationcited by 5
  29. 2026Graph World Models: Concepts, Taxonomy, and Future Directionscited by 5
  30. 2026Code-as-Room: Generating 3D Rooms from Top-Down View Images via Agentic Code Synthesiscited by 5
  31. 2024Layout Generation Agents with Large Language Modelscited by 4
  32. 2025ScanEdit: Hierarchically-Guided Functional 3D Scan Editingcited by 4
  33. 2025HLG: Comprehensive 3D Room Construction via Hierarchical Layout Generationcited by 4
  34. 2026Imagine a City: CityGenAgent for Procedural 3D City Generationcited by 4
  35. 2026FlowScene: Style-Consistent Indoor Scene Generation with Multimodal Graph Rectified Flowcited by 4
  36. 20253D-Generalist: Self-Improving Vision-Language-Action Models for Crafting 3D Worldscited by 3
  37. 20253D Software Synthesis Driven by Constraint-Expressive Intermediate Representationcited by 3
  38. 2026Edit-As-Act: Goal-Regressive Planning for Open-Vocabulary 3D Indoor Scene Editingcited by 3

50 most cited of 81 All 81 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025M3DLayout: A Multi-Source Dataset of 3D Indoor Layouts and Structured Descriptions for 3D Generation
  2. 2025ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editing
  3. 2026CasLayout: Cascaded 3D Layout Diffusion for Indoor Scene Synthesis with Implicit Relation Modeling
  4. 2025Decorum: A Language-Based Approach For Style-Conditioned Synthesis of Indoor 3D Scenes
  5. 2026Global Graph-Validated Optimization for VLM-based 3D Indoor Scene Generation
  6. 2026HDSL: A Hierarchical Domain-Specific Language for Structured 3D Indoor Scene Generation and Localized Editing with LLM Agents
  7. 2026SDesc3D: Towards Layout-Aware 3D Indoor Scene Generation from Short Descriptions
  8. 2024SceneCraft: Layout-Guided 3D Scene Generation