mlsanityData to 7 Oct 2026

RoboTHOR: An Open Simulation-to-Real Embodied AI Platform

arXiv 2004.06799 ↗First posted 14 Apr 2020cs.CV cs.ROcited by 190

Not on the map. Why?

Abstract (arXiv)

Visual recognition ecosystems (e.g. ImageNet, Pascal, COCO) have undeniably played a prevailing role in the evolution of modern computer vision. We argue that interactive and embodied visual AI has reached a stage of development similar to visual recognition prior to the advent of these ecosystems. Recently, various synthetic environments have been introduced to facilitate research in embodied AI. Notwithstanding this progress, the crucial question of how well models trained in simulation generalize to reality has remained largely unanswered. The creation of a comparable ecosystem for simulation-to-real embodied AI presents many challenges: (1) the inherently interactive nature of the problem, (2) the need for tight alignments between real and simulated worlds, (3) the difficulty of replicating physical conditions for repeatable experiments, (4) and the associated cost. In this paper, we introduce RoboTHOR to democratize research in interactive and embodied visual AI. RoboTHOR offers a framework of simulated environments paired with physical counterparts to systematically explore and overcome the challenges of simulation-to-real transfer, and a platform where researchers across the globe can remotely test their embodied models in the physical world. As a first benchmark, our experiments show there exists a significant gap between the performance of models trained in simulation when they are tested in both simulations and their carefully constructed physical analogs. We hope that RoboTHOR will spur the next stage of evolution in embodied computer vision. RoboTHOR can be accessed at the following link: https://ai2thor.allenai.org/robothor

Key facts. Cited by 190 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 31; 2022: 29; 2023: 30; 2024: 37; 2025: 38; 2026: 25.

190 papers.

Sort:

By area:

  1. 2021Domain Generalization: A Surveycited by 760
  2. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  3. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  4. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  5. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
  6. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
  7. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  8. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  9. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  10. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 162
  11. 2021Episodic Transformer for Vision-and-Language Navigationcited by 155
  12. 2022Ditto: Building Digital Twins of Articulated Objects from Interactioncited by 152
38 more
  1. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 126
  2. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 125
  3. 2024PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AIcited by 116
  4. 2021Using Shape to Categorize: Low-Shot Learning with an Explicit Shape Biascited by 115
  5. 2022Habitat-Matterport 3D Semantics Datasetcited by 109
  6. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 106
  7. 2023HomeRobot: Open-Vocabulary Mobile Manipulationcited by 104
  8. 2021Visual Room Rearrangementcited by 101
  9. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 93
  10. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
  11. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  12. 2021VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehiclescited by 81
  13. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 80
  14. 2023Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigationcited by 74
  15. 2024HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigationcited by 71
  16. 2025SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learningcited by 69
  17. 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenescited by 68
  18. 2021ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representationscited by 67
  19. 2021The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AIcited by 64
  20. 2022Target-Driven Structured Transformer Planner for Vision-Language Navigationcited by 58
  21. 2025UniGoal: Towards Universal Zero-shot Goal-oriented Navigationcited by 56
  22. 2024FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuningcited by 54
  23. 2022Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instancescited by 53
  24. 2021RobustNav: Towards Benchmarking Robustness in Embodied Navigationcited by 52
  25. 2023Principles and Guidelines for Evaluating Social Robot Navigation Algorithmscited by 52
  26. 2023SPOC: Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real Worldcited by 47
  27. 2022Retrospectives on the Embodied AI Workshopcited by 46
  28. 2025AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real Worldcited by 46
  29. 2022IFOR: Iterative Flow Minimization for Robotic Object Rearrangementcited by 42
  30. 2022NeRF2Real: Sim2real Transfer of Vision-guided Bipedal Motion Skills using Neural Radiance Fieldscited by 41
  31. 2023Learning Navigational Visual Representations with Semantic Map Supervisioncited by 40
  32. 2022Reinforced Structured State-Evolution for Vision-Language Navigationcited by 36
  33. 20243D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucinationcited by 36
  34. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  35. 2021Hierarchical Object-to-Zone Graph for Object Navigationcited by 34
  36. 2023Learning Human-to-Robot Handovers from Point Cloudscited by 34
  37. 2024CogNav: Cognitive Process Modeling for Object Goal Navigation with LLMscited by 32
  38. 2025UAV-ON: A Benchmark for Open-World Object Goal Navigation with Aerial Agentscited by 31

50 most cited of 190 All 190 (JSON)