mlsanityData to 7 Oct 2026

Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation

arXiv 2306.11290 ↗First posted 20 Jun 2023cs.CVcited by 162

Not on the map. Why?

Abstract (arXiv)

We contribute the Habitat Synthetic Scene Dataset, a dataset of 211 high-quality 3D scenes, and use it to test navigation agent generalization to realistic 3D environments. Our dataset represents real interiors and contains a diverse set of 18,656 models of real-world objects. We investigate the impact of synthetic 3D scene dataset scale and realism on the task of training embodied agents to find and navigate to objects (ObjectGoal navigation). By comparing to synthetic 3D scene datasets from prior work, we find that scale helps in generalization, but the benefits quickly saturate, making visual fidelity and correlation to real-world scenes more important. Our experiments show that agents trained on our smaller-scale dataset can match or outperform agents trained on much larger datasets. Surprisingly, we observe that agents trained on just 122 scenes from our dataset outperform agents trained on 10,000 scenes from the ProcTHOR-10K dataset in terms of zero-shot generalization in real-world scanned environments.

Key facts. Cited by 162 papers in the map (data through 7 Oct 2026). Builds on 32 of its 52 references that are in the map. First posted on arXiv 20 Jun 2023.

Builds on

32 of 52 references.

References · 32

  1. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  2. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  3. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  4. 2018Demystifying MMD GANscited by 1,268
  5. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  6. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  7. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  8. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  9. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  10. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  11. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  12. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
20 more
  1. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  2. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  3. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 306
  4. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
  5. 2020Neural Topological SLAM for Visual Navigationcited by 193
  6. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  7. 2020RoboTHOR: An Open Simulation-to-Real Embodied AI Platformcited by 190
  8. 2020Rearrangement: A Challenge for Embodied AIcited by 153
  9. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
  10. 2022Habitat-Matterport 3D Semantics Datasetcited by 109
  11. 2022Navigating to Objects in the Real Worldcited by 104
  12. 2021Visual Room Rearrangementcited by 101
  13. 2023PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNavcited by 91
  14. 2018Visual Representations for Semantic Target Driven Navigationcited by 89
  15. 2021Auxiliary Tasks and Exploration Enable ObjectNavcited by 73
  16. 2022Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigationcited by 66
  17. 2020Bi-directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agentscited by 33
  18. 2022Rethinking Sim2Real: Lower Fidelity Simulation Leads to Higher Sim2Real Transfer in Navigationcited by 28
  19. 2022VER: Scaling On-Policy RL Leads to the Emergence of Navigation in Embodied Rearrangementcited by 18
  20. 2023Learning Agile, Vision-based Drone Flight: from Simulation to Realitycited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 30; 2025: 62; 2026: 66.

162 papers.

Sort:

By area:

  1. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  2. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
  3. 2025SAM 3D: 3Dfy Anything in Imagescited by 230
  4. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  5. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
  6. 2025Native and Compact Structured Latents for 3D Generationcited by 157
  7. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 146
  8. 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 133
  9. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 130
  10. 2024Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generationcited by 129
  11. 2024PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AIcited by 116
  12. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 106
38 more
  1. 2025Hi3DGen: High-fidelity 3D Geometry Generation from Images via Normal Bridgingcited by 95
  2. 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 81
  3. 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMscited by 72
  4. 2024HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigationcited by 71
  5. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 66
  6. 2025Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generationcited by 65
  7. 2025SpatialLM: Training Large Language Models for Structured Indoor Modelingcited by 63
  8. 2025SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planningcited by 62
  9. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 62
  10. 2024VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Modelcited by 55
  11. 2024PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigatorscited by 55
  12. 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agentscited by 55
  13. 2025SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agentcited by 53
  14. 2025Amodal3R: Amodal 3D Reconstruction from Occluded 2D Imagescited by 52
  15. 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Passcited by 49
  16. 2025NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidancecited by 46
  17. 2024Vision-Language Models Provide Promptable Representations for Reinforcement Learningcited by 41
  18. 2026SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenescited by 39
  19. 2024ImagineNav: Prompting Vision-Language Models as Embodied Navigator through Scene Imaginationcited by 38
  20. 2024Assessment of Multimodal Large Language Models in Alignment with Human Valuescited by 36
  21. 2025MetaScenes: Towards Automated Replica Creation for Real-world 3D Scanscited by 31
  22. 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigationcited by 30
  23. 2024EMOS: Embodiment-aware Heterogeneous Multi-robot Operating System with LLM Agentscited by 24
  24. 2024Embodied VideoAgent: Persistent Memory from Egocentric Videos and Embodied Sensors Enables Dynamic Scene Understandingcited by 24
  25. 2025Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluationcited by 23
  26. 2024S2O: Static to Openable Enhancement for Articulated 3D Objectscited by 22
  27. 2025HSM: Hierarchical Scene Motifs for Multi-Scale Indoor Scene Generationcited by 22
  28. 2025DIPO: Dual-State Images Controlled Articulated Object Generation Powered by Diverse Datacited by 21
  29. 2025UniLat3D: Geometry-Appearance Unified Latents for Single-Stage 3D Generationcited by 20
  30. 2024ReALFRED: An Embodied Instruction Following Benchmark in Photo-Realistic Environmentscited by 16
  31. 2024SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of Expertscited by 16
  32. 2024Survey on Modeling of Human-made Articulated Objectscited by 14
  33. 2025Pandora3D: A Comprehensive Framework for High-Quality 3D Shape and Texture Generationcited by 14
  34. 2025WorldGrow: Generating Infinite 3D Worldcited by 14
  35. 2025FloorplanQA: A Benchmark for Spatial Reasoning in LLMs using Structured Representationscited by 13
  36. 2024LLM-enhanced Scene Graph Learning for Household Rearrangementcited by 12
  37. 2025MoMa-Kitchen: A 100K+ Benchmark for Affordance-Grounded Last-Mile Navigation in Mobile Manipulationcited by 12
  38. 2024Pre-trained Text-to-Image Diffusion Models Are Versatile Representation Learners for Controlcited by 11

50 most cited of 162 All 162 (JSON)