# InternScenes: A Large-scale Simulatable Indoor Scene Dataset with Realistic Layouts

arXiv: https://arxiv.org/abs/2509.10813 · HTML: https://mlsanity.com/paper/2509.10813/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 42 references that are in the map.
- First posted on arXiv 13 Sep 2025.

## Builds on (most cited first, 30 of 34)

- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI](https://mlsanity.com/paper/2109.08238/) (2021)
- [3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics](https://mlsanity.com/paper/2011.09127/) (2020)
- [ProcTHOR: Large-Scale Embodied AI Using Procedural Generation](https://mlsanity.com/paper/2206.06994/) (2022)
- [DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames](https://mlsanity.com/paper/1911.00357/) (2019)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [Scalable 3D Captioning with Pretrained Models](https://mlsanity.com/paper/2306.07279/) (2023)
- [ATISS: Autoregressive Transformers for Indoor Scene Synthesis](https://mlsanity.com/paper/2110.03675/) (2021)
- [RIO: 3D Object Instance Re-Localization in Changing Indoor Environments](https://mlsanity.com/paper/1908.06109/) (2019)
- [DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis](https://mlsanity.com/paper/2303.14207/) (2023)
- [EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI](https://mlsanity.com/paper/2312.16170/) (2023)
- [Scan2CAD: Learning CAD Model Alignment in RGB-D Scans](https://mlsanity.com/paper/1811.11187/) (2018)
- [Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generation](https://mlsanity.com/paper/2406.11824/) (2024)
- [SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding](https://mlsanity.com/paper/2401.09340/) (2024)
- [PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI](https://mlsanity.com/paper/2404.09465/) (2024)
- [MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation](https://mlsanity.com/paper/2412.03558/) (2024)
- [SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Model](https://mlsanity.com/paper/2403.13064/) (2024)
- [Automated Creation of Digital Cousins for Robust Policy Learning](https://mlsanity.com/paper/2410.07408/) (2024)
- [InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset](https://mlsanity.com/paper/1809.00716/) (2018)

## Built on by (most cited first, 20 of 20)

- [SceneReVis: A Self-Reflective Vision-Grounded Framework for 3D Indoor Scene Synthesis via Multi-Turn RL](https://mlsanity.com/paper/2602.09432/) (2026)
- [Imagine a City: CityGenAgent for Procedural 3D City Generation](https://mlsanity.com/paper/2602.05362/) (2026)
- [MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learning](https://mlsanity.com/paper/2510.03142/) (2025)
- [ArtiWorld: LLM-Driven Articulation of 3D Objects in Scenes](https://mlsanity.com/paper/2511.12977/) (2025)
- [SynthVerse: A Large-Scale Diverse Synthetic Dataset for Point Tracking](https://mlsanity.com/paper/2602.04441/) (2026)
- [Repurposing 3D Generative Model for Autoregressive Layout Generation](https://mlsanity.com/paper/2604.16299/) (2026)
- [FIRE3D: Feed-forward Interactive 3D Scene Reconstruction Within A Minute](https://mlsanity.com/paper/2609.08848/) (2026)
- [NaLA: A 3D Native LLM Layout Agent for High-quality 3D Scene Generation](https://mlsanity.com/paper/2606.29395/) (2026)
- [Pair2Scene: Learning Local Object Relations for Procedural Scene Generation](https://mlsanity.com/paper/2604.11808/) (2026)
- [SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes](https://mlsanity.com/paper/2605.31148/) (2026)
- [Real2Gym: Building Gyms from Videos, Bringing Skills to Robots](https://mlsanity.com/paper/2609.37089/) (2026)
- [Exploring Spatial Intelligence from a Generative Perspective](https://mlsanity.com/paper/2604.20570/) (2026)
- [Orchestrating Spatial Semantics via a Zone-Graph Paradigm for Intricate Indoor Scene Generation](https://mlsanity.com/paper/2605.02537/) (2026)
- [GARDEN: Gravity-Aligned Reconstruction of Disentangled ENvironments from RGB images](https://mlsanity.com/paper/2606.03921/) (2026)
- [HomeWorld: A Unified Floorplan-to-Furnished Framework for Generating Controllable, Densely Interactive Whole-Home Scenes](https://mlsanity.com/paper/2606.06390/) (2026)
- [SceneMosaic: Efficient and Diverse Simulation-Ready Scene Generation via Hybrid Agentic Layout Evolution](https://mlsanity.com/paper/2609.05594/) (2026)
- [EgoPathBench: Evaluating Zero-Shot Egocentric Waypoint Decision-Making in Vision-Language Models](https://mlsanity.com/paper/2609.16610/) (2026)
- [GATOR: Generative and Agentic 3D Object Reconstruction From Casual Images](https://mlsanity.com/paper/2610.11215/) (2026)
- [DR-APO: Decoupled-Representation Advantage-Guided Policy Optimization from Continuous Control to Embodied Navigation](https://mlsanity.com/paper/2610.16618/) (2026)
- [Code2Worlds-2: Physically Plausible Agent Harness for Embodied World Simulation](https://mlsanity.com/paper/2610.17651/) (2026)

## Abstract (arXiv)

The advancement of Embodied AI heavily relies on large-scale, simulatable 3D scene datasets characterized by scene diversity and realistic layouts. However, existing datasets typically suffer from limitations in data scale or diversity, sanitized layouts lacking small items, and severe object collisions. To address these shortcomings, we introduce InternScenes, a novel large-scale simulatable indoor scene dataset comprising approximately 40,000 diverse scenes by integrating three disparate scene sources, real-world scans, procedurally generated scenes, and designer-created scenes, including 1.96M 3D objects and covering 15 common scene types and 288 object classes. We particularly preserve massive small items in the scenes, resulting in realistic and complex layouts with an average of 41.5 objects per region. Our comprehensive data processing pipeline ensures simulatability by creating real-to-sim replicas for real-world scans, enhances interactivity by incorporating interactive objects into these scenes, and resolves object collisions by physical simulations. We demonstrate the value of InternScenes with two benchmark applications: scene layout generation and point-goal navigation. Both show the new challenges posed by the complex and realistic layouts. More importantly, InternScenes paves the way for scaling up the model training for both tasks, making the generation and navigation in such complex scenes possible. We commit to open-sourcing the data, models, and benchmarks to benefit the whole community.