mlsanityData to 14 Oct 2026

AI2-THOR: An Interactive 3D Environment for Visual AI

arXiv 1712.05474 ↗First posted 14 Dec 2017cs.CV cs.AI cs.LGcited by 869

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

We introduce The House Of inteRactions (THOR), a framework for visual AI research, available at http://ai2thor.allenai.org. AI2-THOR consists of near photo-realistic 3D indoor scenes, where AI agents can navigate in the scenes and interact with objects to perform tasks. AI2-THOR enables research in many different domains including but not limited to deep reinforcement learning, imitation learning, learning by interaction, planning, visual question answering, unsupervised representation learning, object detection and segmentation, and learning models of cognition. The goal of AI2-THOR is to facilitate building visually intelligent models and push the research forward in this domain.

Key facts. Cited by 869 papers in the map (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 2 of its 29 references that are in the map. First posted on arXiv 14 Dec 2017.

Builds on

2 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2017Matterport3D: Learning from RGB-D Data in Indoor Environments18 Sep

Other references · 1

  1. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 2; 2019: 1; 2020: 4; 2021: 102; 2022: 110; 2023: 128; 2024: 154; 2025: 181; 2026: 187.

869 papers.

Sort:

By area:

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  3. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  4. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  5. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  6. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  7. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  8. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
  9. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
  10. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
  11. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  12. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
38 more
  1. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  2. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
  3. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 439
  4. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
  5. 2022Kubric: A scalable dataset generatorcited by 381
  6. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
  7. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
  8. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  9. 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
  10. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
  11. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
  12. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  13. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 268
  14. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
  15. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 250
  16. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 239
  17. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 237
  18. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
  19. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 228
  20. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 221
  21. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 221
  22. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 197
  23. 2021TEACh: Task-driven Embodied Agents that Chatcited by 197
  24. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 194
  25. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  26. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  27. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 178
  28. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 177
  29. 2023REFLECT: Summarizing Robot Experiences for Failure Explanation and Correctioncited by 175
  30. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
  31. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 175
  32. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 170
  33. 2023SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Taskscited by 168
  34. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 167
  35. 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 166
  36. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  37. 2021BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environmentscited by 165
  38. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 164

50 most cited of 869 All 869 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024THOR: Text to Human-Object Interaction Diffusion via Relation Intervention
  2. 2026AgentGarten: Code Worlds for Evolving Agents
  3. 2026SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes
  4. 2026From Perception to Action: An Interactive Benchmark for Vision Reasoning
  5. 2025NEBULA: Do We Evaluate Vision-Language-Action Agents Correctly?
  6. 2021Creating Multimodal Interactive Agents with Imitation and Self-Supervised Learning
  7. 2026Beyond Placement and Articulation: Usage-Driven Code Scenes for Embodied Interaction
  8. 2026SceneCode: Executable World Programs for Editable Indoor Scenes with Articulated Objects