mlsanityData to 14 Oct 2026

Foundation Models in Robotics: Applications, Challenges, and the Future

arXiv 2312.07843 ↗First posted 13 Dec 2023cs.ROcited by 216

Not on the map. Why?

Cited by 38 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We survey applications of pretrained foundation models in robotics. Traditional deep learning models in robotics are trained on small datasets tailored for specific tasks, which limits their adaptability across diverse applications. In contrast, foundation models pretrained on internet-scale data appear to have superior generalization capabilities, and in some instances display an emergent ability to find zero-shot solutions to problems that are not present in the training data. Foundation models may hold the potential to enhance various components of the robot autonomy stack, from perception to decision-making and control. For example, large language models can generate code or provide common sense reasoning, while vision-language models enable open-vocabulary visual recognition. However, significant open research challenges remain, particularly around the scarcity of robot-relevant training data, safety guarantees and uncertainty quantification, and real-time execution. In this survey, we study recent papers that have used or built foundation models to solve robotics problems. We explore how foundation models contribute to improving robot capabilities in the domains of perception, decision-making, and control. We discuss the challenges hindering the adoption of foundation models in robot autonomy and provide opportunities and potential pathways for future advancements. The GitHub project corresponding to this paper (Preliminary release. We are committed to further enhancing and updating this work to ensure its quality and relevance) can be found here: https://github.com/robotics-survey/Awesome-Robotics-Foundation-Models

Key facts. Cited by 216 papers in the map (data through 14 Oct 2026). First posted on arXiv 13 Dec 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 53; 2025: 104; 2026: 59.

216 papers.

Sort:

By area:

  1. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 315
  2. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  3. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 190
  4. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learningcited by 151
  5. 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 108
  6. 2025TesserAct: Learning 4D Embodied World Modelscited by 94
  7. 2024Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistancecited by 87
  8. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
  9. 20253D and 4D World Modeling: A Surveycited by 67
  10. 2025Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoningcited by 63
  11. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 61
  12. 2024FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuningcited by 54
38 more
  1. 2024Towards Efficient LLM Grounding for Embodied Multi-Agent Collaborationcited by 45
  2. 2024CoNVOI: Context-aware Navigation using Vision Language Models in Outdoor and Indoor Environmentscited by 43
  3. 2025Diffusion Models for Robotic Manipulation: A Surveycited by 41
  4. 2024Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4Vcited by 40
  5. 2024Splat-MOVER: Multi-Stage, Open-Vocabulary Robotic Manipulation via Editable Gaussian Splattingcited by 38
  6. 2025BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimizationcited by 38
  7. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 36
  8. 2025VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Modelscited by 36
  9. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  10. 2025Generating Robot Constitutions & Benchmarks for Semantic Safetycited by 33
  11. 2026TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Roboticscited by 33
  12. 2024The Essential Role of Causality in Foundation World Models for Embodied AIcited by 32
  13. 2024TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulationcited by 30
  14. 2025What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Modelscited by 29
  15. 2025RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Modelscited by 28
  16. 2024M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenescited by 27
  17. 2025Large Language Models for Multi-Robot Systems: A Surveycited by 27
  18. 2024Task Success is not Enough: Investigating the Use of Video-Language Models as Behavior Critics for Catching Undesirable Agent Behaviorscited by 25
  19. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24
  20. 2025Inference-Time Enhancement of Generative Robot Policies via Predictive World Modelingcited by 24
  21. 2025ManipBench: Benchmarking Vision-Language Models for Low-Level Robot Manipulationcited by 24
  22. 2025Embodied AI: From LLMs to World Modelscited by 24
  23. 2025UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulationcited by 23
  24. 2024BlenderAlchemy: Editing 3D Graphics with Vision-Language Modelscited by 22
  25. 2025Being-0: A Humanoid Robotic Agent with Vision-Language Models and Modular Skillscited by 22
  26. 2024VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulationcited by 21
  27. 2025Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interactioncited by 19
  28. 2025Seeing to Act, Prompting to Specify: A Bayesian Factorization of Vision Language Action Policycited by 18
  29. 2024PWM: Policy Learning with Multi-Task World Modelscited by 17
  30. 2024LangSurf: Language-Embedded Surface Gaussians for 3D Scene Understandingcited by 17
  31. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  32. 2025Bridging the Semantic-Action Gap in Visual Token Pruning for Efficient VLA Inferencecited by 17
  33. 2024Verifiably Following Complex Robot Instructions with Foundation Modelscited by 16
  34. 2025π_textttRL: Online RL Fine-tuning for Flow-based Vision-Language-Action Modelscited by 16
  35. 2024Evaluating Uncertainty-based Failure Detection for Closed-Loop LLM Plannerscited by 15
  36. 2025RoboDexVLM: Visual Language Model-Enabled Task Planning and Motion Control for Dexterous Robot Manipulationcited by 15
  37. 2024A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Taskscited by 14
  38. 2025A Survey on Imitation Learning for Contact-Rich Tasks in Roboticscited by 14

50 most cited of 216 All 216 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Foundation Models in Robotics: A Comprehensive Review of Methods, Models, Datasets, Challenges and Future Research Directions
  2. 2025Foundation Model Driven Robotics: A Comprehensive Review
  3. 2022Policy Adaptation from Foundation Model Feedback
  4. 2024A Survey for Foundation Models in Autonomous Driving
  5. 2025From Grounding to Manipulation: Case Studies of Foundation Model Integration in Embodied Robotic Systems
  6. 2025SPEAR-1: Scaling Beyond Robot Demonstrations via 3D Understanding
  7. 2026Are Foundation Models the Route to Full-Stack Transfer in Robotics? ↗
  8. 2025Can Pretrained Vision-Language Embeddings Alone Guide Robot Navigation?