mlsanityData to 7 Oct 2026

Habitat: A Platform for Embodied AI Research

arXiv 1904.01201 ↗First posted 2 Apr 2019cs.CV cs.AI cs.CLcited by 1,163

On the map: World models → World models for navigation

Abstract (arXiv)

We present Habitat, a platform for research in embodied artificial intelligence (AI). Habitat enables training embodied agents (virtual robots) in highly efficient photorealistic 3D simulation. Specifically, Habitat consists of: (i) Habitat-Sim: a flexible, high-performance 3D simulator with configurable agents, sensors, and generic 3D dataset handling. Habitat-Sim is fast -- when rendering a scene from Matterport3D, it achieves several thousand frames per second (fps) running single-threaded, and can reach over 10,000 fps multi-process on a single GPU. (ii) Habitat-API: a modular high-level library for end-to-end development of embodied AI algorithms -- defining tasks (e.g., navigation, instruction following, question answering), configuring, training, and benchmarking embodied agents. These large-scale engineering contributions enable us to answer scientific questions requiring experiments that were till now impracticable or 'merely' impractical. Specifically, in the context of point-goal navigation: (1) we revisit the comparison between learning and SLAM approaches from two recent works and find evidence for the opposite conclusion -- that learning outperforms SLAM if scaled to an order of magnitude more experience than previous investigations, and (2) we conduct the first cross-dataset generalization experiments {train, test} x {Matterport3D, Gibson} for multiple sensors {blind, RGB, RGBD, D} and find that only agents with depth (D) sensors generalize across datasets. We hope that our open-source platform and these findings will advance research in embodied AI.

Key facts. Cited by 1,163 papers in the map (data through 7 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 3 of its 35 references that are in the map. First posted on arXiv 2 Apr 2019.

Builds on

3 of 35 references.

References · 3

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  3. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 4; 2021: 131; 2022: 168; 2023: 187; 2024: 189; 2025: 238; 2026: 246.

1,163 papers.

Sort:

By area:

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  2. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  3. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  4. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  5. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  6. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  7. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  8. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  9. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  10. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  11. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  12. 2023Learning Interactive Real-World Simulatorscited by 440
38 more
  1. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
  2. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 411
  3. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  4. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  5. 2022Kubric: A scalable dataset generatorcited by 373
  6. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  7. 2022Visual Language Maps for Robot Navigationcited by 362
  8. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 356
  9. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  10. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  11. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
  12. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  13. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  14. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 296
  15. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  16. 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 292
  17. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  18. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  19. 20243D Reconstruction with Spatial Memorycited by 269
  20. 2024Navigation World Modelscited by 262
  21. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
  22. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
  23. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
  24. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 242
  25. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 239
  26. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 237
  27. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  28. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  29. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  30. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 215
  31. 2023ViNT: A Foundation Model for Visual Navigationcited by 210
  32. 2021Stochastic Scene-Aware Motion Predictioncited by 203
  33. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
  34. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  35. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  36. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
  37. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 191
  38. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191

50 most cited of 1,163 All 1,163 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environment
  2. 2026EmbodiedGen V2: An Agentic, Simulation-Ready 3D World Engine for Embodied AI
  3. 2026SAGE: Scalable Agentic 3D Scene Generation for Embodied AI
  4. 2025Embodied Spatial Intelligence: from Implicit Scene Modeling to Spatial Reasoning
  5. 2026InHabit: Leveraging Image Foundation Models for Scalable 3D Human Placement
  6. 2026Embodied-BenchClaw: An Autonomous Multi-Agent System for Embodied Spatial Intelligence Benchmark Construction
  7. 2025Vision to Geometry: 3D Spatial Memory for Sequential Embodied MLLM Reasoning and Exploration
  8. 2026Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos