mlsanityData to 14 Oct 2026

Retrospectives on the Embodied AI Workshop

arXiv 2210.06849 ↗First posted 13 Oct 2022cs.CVcited by 46

Not on the map. Why?

Abstract (arXiv)

We present a retrospective on the state of Embodied AI research. Our analysis focuses on 13 challenges presented at the Embodied AI Workshop at CVPR. These challenges are grouped into three themes: (1) visual navigation, (2) rearrangement, and (3) embodied vision-and-language. We discuss the dominant datasets within each theme, evaluation metrics for the challenges, and the performance of state-of-the-art models. We highlight commonalities between top approaches to the challenges and identify potential future directions for Embodied AI research.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). Builds on 169 of its 220 references that are in the map. First posted on arXiv 13 Oct 2022.

Builds on

169 of 220 references.

References · 169

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  9. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  10. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  11. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  12. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
38 more
  1. 2018World Modelscited by 1,513
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  3. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  4. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  5. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  6. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
  7. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  8. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  9. 2020Mastering Atari with Discrete World Modelscited by 1,007
  10. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  11. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  12. 2022A Generalist Agentcited by 808
  13. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
  14. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  15. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  16. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 629
  17. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 622
  18. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 612
  19. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
  20. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  21. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 583
  22. 2018On Evaluation of Embodied Navigation Agentscited by 547
  23. 2020Rescaling Egocentric Visioncited by 540
  24. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
  25. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  26. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
  27. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  28. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
  29. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  30. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  31. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  32. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  33. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
  34. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 421
  35. 2018VirtualHome: Simulating Household Activities via Programscited by 415
  36. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  37. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
  38. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380

50 most cited of 169 All 169 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 7; 2023: 17; 2024: 10; 2025: 12.

46 papers.

Sort:

By area:

  1. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  2. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 237
  3. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 218
  4. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
  5. 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenescited by 69
  6. 2023Principles and Guidelines for Evaluating Social Robot Navigation Algorithmscited by 56
  7. 2022Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instancescited by 55
  8. 2023Navigating to Objects Specified by Imagescited by 51
  9. 2023Learning Human-to-Robot Handovers from Point Cloudscited by 34
  10. 2024OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Roboticscited by 34
  11. 2025Vid2Sim: Realistic and Interactive Simulation from Video for Urban Navigationcited by 30
  12. 2023Pretrained Language Models as Visual Planners for Human Assistancecited by 28
34 more
  1. 2023Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approachcited by 28
  2. 2022Last-Mile Embodied Visual Navigationcited by 25
  3. 2022PEANUT: Predicting and Navigating to Unseen Targetscited by 22
  4. 2023Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learningcited by 22
  5. 2024MetaUrban: An Embodied AI Simulation Platform for Urban Micromobilitycited by 21
  6. 2023Scene-Driven Multimodal Knowledge Graph Construction for Embodied AIcited by 17
  7. 2025Enter the Mind Palace: Reasoning and Planning for Long-term Active Embodied Question Answeringcited by 14
  8. 2024Constrained Human-AI Cooperation: An Inclusive Embodied Social Intelligence Challengecited by 13
  9. 2023MOPA: Modular Object Navigation with PointGoal Agentscited by 11
  10. 2023AutoNeRF: Training Implicit Scene Representations with Autonomous Agentscited by 10
  11. 2023HabiCrowd: A High Performance Simulator for Crowd-Aware Visual Navigationcited by 9
  12. 2023Exploitation-Guided Exploration for Semantic Embodied Navigationcited by 9
  13. 2024Towards Open-World Mobile Manipulation in Homes: Lessons from the Neurips 2023 HomeRobot Open Vocabulary Mobile Manipulation Challengecited by 9
  14. 2024Task-oriented Sequential Grounding and Navigation in 3D Scenescited by 9
  15. 2024Sim2Real Transfer for Audio-Visual Navigation with Frequency-Adaptive Acoustic Field Predictioncited by 7
  16. 2022Exploiting Proximity-Aware Tasks for Embodied Social Navigationcited by 6
  17. 2022Learning-Augmented Model-Based Planning for Visual Explorationcited by 5
  18. 2024Following the Human Thread in Social Navigationcited by 4
  19. 2025DAgger Diffusion Navigation: DAgger Boosted Diffusion Policy for Vision-Language Navigationcited by 4
  20. 2024Fusion Dynamical Systems with Machine Learning in Imitation Learning: A Comprehensive Overviewcited by 3
  21. 2025UrbanVLA: A Vision-Language-Action Model for Urban Micromobilitycited by 3
  22. 2022Comparison of Model-Free and Model-Based Learning-Informed Planning for PointGoal Navigationcited by 2
  23. 2025Advancing Embodied Intelligence in Robotic-Assisted Endovascular Procedures: A Systematic Review of AI Solutionscited by 2
  24. 2025Situat3DChange: Situated 3D Change Understanding Dataset for Multimodal Large Language Modelcited by 2
  25. 2025LagMemo: Language 3D Gaussian Splatting Memory for Multi-modal Open-vocabulary Multi-goal Visual Navigationcited by 2
  26. 2023SegmATRon: Embodied Adaptive Semantic Segmentation for Indoor Environmentcited by 1
  27. 2024Temporal and Semantic Evaluation Metrics for Foundation Models in Post-Hoc Analysis of Robotic Sub-taskscited by 1
  28. 2024Multimodal Datasets and Benchmarks for Reasoning about Dynamic Spatio-Temporality in Everyday Environmentscited by 1
  29. 2025Embodied AI in Machine Learning -- is it Really Embodied?cited by 1
  30. 2025MLFM: Multi-Layered Feature Maps for Richer Language Understanding in Zero-Shot Semantic Navigationcited by 1
  31. 2023Glamour muscles: why having a body is not what it means to be embodiedcited by 0
  32. 2025HomeEmergency -- Using Audio to Find and Respond to Emergencies in the Homecited by 0
  33. 2025Improving Cooperation in Collaborative Embodied AIcited by 0
  34. 2025FOSSIL: Harnessing Feedback on Suboptimal Samples for Data-Efficient Generalisation with Imitation Learning for Embodied Vision-and-Language Taskscited by 0