MANSION: Multi-floor lANguage-to-3D Scene generatIOn for loNg-horizon tasks
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Real-world robotic tasks are long-horizon and often span multiple floors, demanding rich spatial reasoning. However, existing embodied benchmarks are largely confined to single-floor in-house environments, failing to reflect the complexity of real-world tasks. We introduce MANSION, the first language-driven framework for generating building-scale, multi-floor 3D environments. Being aware of vertical structural constraints, MANSION generates realistic, navigable whole-building structures with diverse, human-friendly scenes, enabling the development and evaluation of cross-floor long-horizon tasks. Building on this framework, we release MansionWorld, a dataset of over 1,000 diverse buildings ranging from hospitals to offices, alongside a Task-Semantic Scene Editing Agent that customizes these environments using open-vocabulary commands to meet specific user needs. Benchmarking reveals that state-of-the-art agents degrade sharply in our settings, establishing MANSION as a critical testbed for the next generation of spatial reasoning and planning.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 42 of its 52 references that are in the map. First posted on arXiv 12 Mar 2026.
Builds on
42 of 52 references.
Closest ancestors · 3
Same direction, newest first
- 2024BUMBLE: Unifying Reasoning and Acting with Vision-Language Models for Building-wide Mobile Manipulation8 Oct
- 2024ReMEmbR: Building and Reasoning Over Long-Horizon Spatio-Temporal Memory for Robot Navigation20 Sep
- 2022Inner Monologue: Embodied Reasoning through Planning with Language Models12 Jul
Other references · 39
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
- 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
- 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
27 more
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
- 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
- 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
- 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
- 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
- 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
- 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
- 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
- 2021TEACh: Task-driven Embodied Agents that Chatcited by 194
- 2024BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulationcited by 140
- 2019Object-driven Text-to-Image Synthesis via Adversarial Trainingcited by 117
- 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 81
- 2022HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoisingcited by 72
- 2024GRUtopia: Dream General Robots in a City at Scalecited by 54
- 2025SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agentcited by 53
- 2024Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4Vcited by 40
- 2024SceneCraft: Layout-Guided 3D Scene Generationcited by 40
- 2025R-VLM: Region-Aware Vision Language Model for Precise GUI Groundingcited by 20
- 2023Tell2Design: A Dataset for Language-Guided Floor Plan Generationcited by 19
- 2024MSD: A Benchmark Dataset for Floor Plan Generation of Building Complexescited by 13
- 2023ESceme: Vision-and-Language Navigation with Episodic Scene Memorycited by 9
- 2024GSDiff: Synthesizing Vector Floorplans via Geometry-enhanced Structural Graph Generationcited by 8
- 2025ResPlan: A Large-Scale Vector-Graph Dataset of 17,000 Residential Floor Planscited by 7
- 2025STMA: A Spatio-Temporal Memory Agent for Long-Horizon Embodied Task Planningcited by 5
- 2024ChatHouseDiffusion: Prompt-Guided Generation and Editing of Floor Planscited by 4
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026WorldClaw: Agentic 3D Open-World Generation at Scalecited by 6
- 2026FIRE3D: Feed-forward Interactive 3D Scene Reconstruction Within A Minutecited by 3
- 2026PlanCraft: Sketch, Refine, and Furnish for Architect-Inspired Progressive 3D Residential Scene Generationcited by 1
- 2026GIF: Agentic Generation of Interactive and Functional Object Compositions for Robot Learningcited by 1
- 2026To See a World in a Living Context: Unified Indoor-Outdoor Urban World Generationcited by 0
- 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 0
- 2026ProxyBuild: Text-Guided Structured 3D Building Generation with Mesh-Anchored Procedural Proxiescited by 0
- 2026Affordance-Conditioned Decision Making: Bridging the Semantic-Spatial Gap in Zero-Shot Cross-Floor Vision-and-Language Navigationcited by 0
- 2026Affordance-Conditioned Decision Making: Bridging the Semantic-Spatial Gap in Zero-Shot Cross-Floor Vision-and-Language Navigation26 Sep 2026
- 2026ProxyBuild: Text-Guided Structured 3D Building Generation with Mesh-Anchored Procedural Proxies20 Sep 2026
- 2026FIRE3D: Feed-forward Interactive 3D Scene Reconstruction Within A Minute8 Sep 2026
- 2026Agentic Visual Generation: From Generative Models to Agentic Control6 Sep 2026
- 2026GIF: Agentic Generation of Interactive and Functional Object Compositions for Robot Learning5 Sep 2026
- 2026To See a World in a Living Context: Unified Indoor-Outdoor Urban World Generation6 Aug 2026
- 2026WorldClaw: Agentic 3D Open-World Generation at Scale5 Aug 2026
- 2026PlanCraft: Sketch, Refine, and Furnish for Architect-Inspired Progressive 3D Residential Scene Generation26 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes
- 2026SceneFoundry: Generating Interactive Infinite 3D Worlds
- 2026Tokenization Allows Multimodal Large Language Models to Understand, Generate and Edit Architectural Floor Plans
- 2026HomeWorld: A Unified Floorplan-to-Furnished Framework for Generating Controllable, Densely Interactive Whole-Home Scenes
- 2026SceneCode: Executable World Programs for Editable Indoor Scenes with Articulated Objects
- 2026SceneCritic: A Symbolic Evaluator for 3D Indoor Scene Synthesis
- 2026RoboLayout: Differentiable 3D Scene Generation for Embodied Agents
- 2026D3D-GEN: Robot-Aware Domain-Grounded Interactive 3D World Generation for Social Robotics ↗