mlsanityData to 7 Oct 2026

A Survey of Embodied AI: From Simulators to Research Tasks

arXiv 2103.04918 ↗First posted 8 Mar 2021cs.AI cs.LGcited by 256

Not on the map. Why?

Cited by 27 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

There has been an emerging paradigm shift from the era of "internet AI" to "embodied AI", where AI algorithms and agents no longer learn from datasets of images, videos or text curated primarily from the internet. Instead, they learn through interactions with their environments from an egocentric perception similar to humans. Consequently, there has been substantial growth in the demand for embodied AI simulators to support various embodied AI research tasks. This growing interest in embodied AI is beneficial to the greater pursuit of Artificial General Intelligence (AGI), but there has not been a contemporary and comprehensive survey of this field. This paper aims to provide an encyclopedic survey for the field of embodied AI, from its simulators to its research. By evaluating nine current embodied AI simulators with our proposed seven features, this paper aims to understand the simulators in their provision for use in embodied AI research and their limitations. Lastly, this paper surveys the three main research tasks in embodied AI -- visual exploration, visual navigation and embodied question answering (QA), covering the state-of-the-art approaches, evaluation metrics and datasets. Finally, with the new insights revealed through surveying the field, the paper will provide suggestions for simulator-for-task selections and recommendations for the future directions of the field.

Key facts. Cited by 256 papers in the map (data through 7 Oct 2026). Builds on 100 of its 152 references that are in the map. First posted on arXiv 8 Mar 2021.

Builds on

100 of 152 references.

References · 100

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  7. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  8. 2016OpenAI Gymcited by 1,684
  9. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  10. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  11. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  12. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
38 more
  1. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  2. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  3. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  4. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  5. 2018Exploration by Random Network Distillationcited by 966
  6. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  7. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  8. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  9. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  10. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  11. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  12. 2018On Evaluation of Embodied Navigation Agentscited by 526
  13. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
  14. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  15. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  16. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  17. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  18. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  19. 2016Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Agecited by 405
  20. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
  21. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  22. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  23. 2017Embodied Question Answeringcited by 335
  24. 2016Successor Features for Transfer in Reinforcement Learningcited by 331
  25. 2020Learning to Explore using Active Neural SLAMcited by 308
  26. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
  27. 2019Self-Supervised Exploration via Disagreementcited by 263
  28. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
  29. 2019Vision-and-Dialog Navigationcited by 246
  30. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
  31. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  32. 2017Cognitive Mapping and Planning for Visual Navigationcited by 209
  33. 2018Semi-parametric Topological Memory for Navigationcited by 203
  34. 2020Neural Topological SLAM for Visual Navigationcited by 193
  35. 2020RoboTHOR: An Open Simulation-to-Real Embodied AI Platformcited by 190
  36. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 177
  37. 2017IQA: Visual Question Answering in Interactive Environmentscited by 154
  38. 2018Visual Semantic Navigation using Scene Priorscited by 127

50 most cited of 100 All 100 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 18; 2023: 42; 2024: 53; 2025: 74; 2026: 59.

256 papers.

Sort:

By area:

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  2. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  3. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  4. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  5. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  6. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  7. 2023Tool Learning with Foundation Modelscited by 302
  8. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  9. 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Modelscited by 204
  10. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 201
  11. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  12. 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Modelscited by 158
38 more
  1. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
  2. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  3. 2023MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devicescited by 120
  4. 2022Navigating to Objects in the Real Worldcited by 104
  5. 2025Efficient Reasoning Models: A Surveycited by 93
  6. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  7. 2021Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directionscited by 73
  8. 2024Diffusion Models in Low-Level Vision: A Surveycited by 73
  9. 2024Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulationcited by 71
  10. 2023ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding with GPT and Prototype Guidancecited by 70
  11. 2024Unifying 3D Vision-Language Understanding via Promptable Queriescited by 70
  12. 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenescited by 68
  13. 2023Penetrative AI: Making LLMs Comprehend the Physical Worldcited by 64
  14. 2023TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usagecited by 54
  15. 2023Renderable Neural Radiance Map for Visual Navigationcited by 51
  16. 2023Adaptive Rotated Convolution for Rotated Object Detectioncited by 51
  17. 2024LoTa-Bench: Benchmarking Language-oriented Task Planners for Embodied Agentscited by 51
  18. 2025RLinf: Flexible and Efficient Large-scale Reinforcement Learning via Macro-to-Micro Flow Transformationcited by 51
  19. 2025Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigationcited by 50
  20. 2025TrackVLA: Embodied Visual Tracking in the Wildcited by 46
  21. 2024AI Flow at the Network Edgecited by 43
  22. 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environmentcited by 42
  23. 2024EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agentscited by 41
  24. 2025IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Taskscited by 40
  25. 2023TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systemscited by 39
  26. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  27. 2025UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Makingcited by 36
  28. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 34
  29. 2021Vision-Language Navigation: A Survey and Taxonomycited by 33
  30. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructionscited by 33
  31. 2025Matrix-3D: Omnidirectional Explorable 3D World Generationcited by 33
  32. 2023LLM as A Robotic Brain: Unifying Egocentric Memory and Controlcited by 32
  33. 2025CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Spacecited by 31
  34. 2025A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrievalcited by 31
  35. 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigationcited by 30
  36. 2023Towards Knowledge-driven Autonomous Drivingcited by 29
  37. 2025Shifting AI Efficiency From Model-Centric to Data-Centric Compressioncited by 29
  38. 2023Pretrained Language Models as Visual Planners for Human Assistancecited by 28

50 most cited of 256 All 256 (JSON)