mlsanityData to 7 Oct 2026

A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI

arXiv 2505.01458 ↗First posted 1 May 2025cs.RO cs.AIcited by 16

Not on the map. Why?

Abstract (arXiv)

Navigation and manipulation are core capabilities in Embodied AI, but training agents to perform them directly in the real world is costly, time-consuming, and unsafe. Therefore, sim-to-real transfer has emerged as a key approach, yet the sim-to-real gap persists. This survey examines how physics simulators address this gap by analyzing properties that have received limited attention in prior surveys. We also analyze their features for navigation and manipulation tasks, as well as their hardware requirements. Additionally, we offer a resource with benchmark datasets, metrics, simulation platforms, and methods to help researchers select suitable tools while accounting for hardware constraints.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 205 of its 288 references that are in the map. First posted on arXiv 1 May 2025.

Builds on

205 of 288 references.

References · 205

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  8. 2018Neural Ordinary Differential Equationscited by 3,733
  9. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  10. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  11. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  12. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
38 more
  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  2. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  3. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  4. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  5. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  6. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  7. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  8. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  9. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  10. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  11. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  12. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  13. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  14. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  15. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  16. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  17. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  18. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  19. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  20. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  21. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  22. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  23. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  24. 2019Solving Rubik's Cube with a Robot Handcited by 699
  25. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  26. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  27. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  28. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  29. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  30. 2018Learning Dexterous In-Hand Manipulationcited by 643
  31. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  32. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  33. 2019Three scenarios for continual learningcited by 610
  34. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  35. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  36. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  37. 2023LERF: Language Embedded Radiance Fieldscited by 538
  38. 2018On Evaluation of Embodied Navigation Agentscited by 526

50 most cited of 205 All 205 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 8.

16 papers.

Sort:

By area:

  1. 2025Latent Action Pretraining Through World Modelingcited by 21
  2. 2025Compose Your Policies! Improving Diffusion-based or Flow-based Robot Policies via Test-time Distribution-level Compositioncited by 8
  3. 2025Sari Sandbox: A Virtual Retail Store Environment for Embodied AI Agentscited by 4
  4. 2025Look Beyond: Two-Stage Scene View Generation via Panorama and Video Diffusioncited by 3
  5. 2025Embodied4C: Measuring What Matters for Embodied Vision-Language Navigationcited by 3
  6. 2026MapTab: A Diagnostic Benchmark for Long-Horizon Multi-Criteria Multimodal Reasoning on Heterogeneous Topological Graphscited by 3
  7. 2026LABSHIELD: A Multimodal Benchmark for Safety-Critical Reasoning and Planning in Scientific Laboratoriescited by 3
  8. 2025LagMemo: Language 3D Gaussian Splatting Memory for Multi-modal Open-vocabulary Multi-goal Visual Navigationcited by 2
  9. 2025MG-Nav: Dual-Scale Visual Navigation via Sparse Spatial Memorycited by 2
  10. 2026PhyMix: Towards Physically Consistent Single-Image 3D Indoor Scene Generation with Implicit--Explicit Optimizationcited by 2
  11. 2026Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPOcited by 1
  12. 2026EvoMemNav: Efficient Self-Evolving Fine-Grained Memory for Zero-Shot Embodied Navigationcited by 1
4 more
  1. 2025Fundamentals of Physical AIcited by 0
  2. 2026ClickSeg3D: Few-Click Interactive Segmentation via Semantic Embeddingscited by 0
  3. 2026Synthetic Data Generation and Vision-based Wrinkle and Keypoint Detection for Bimanual Cloth Manipulationcited by 0
  4. 2026RoboSPA: Can VLA Models Go Beyond Simple Scenes and Short-Horizon Tasks?cited by 0