mlsanityData to 7 Oct 2026

Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Review

arXiv 2505.20503 ↗First posted 26 May 2025cs.RO cs.AI cs.CLcited by 3

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Rapid advancements in foundation models, including Large Language Models, Vision-Language Models, Multimodal Large Language Models, and Vision-Language-Action Models, have opened new avenues for embodied AI in mobile service robotics. By combining foundation models with the principles of embodied AI, where intelligent systems perceive, reason, and act through physical interaction, mobile service robots can achieve more flexible understanding, adaptive behavior, and robust task execution in dynamic real-world environments. Despite this progress, embodied AI for mobile service robots continues to face fundamental challenges related to the translation of natural language instructions into executable robot actions, multimodal perception in human-centered environments, uncertainty estimation for safe decision-making, and computational constraints for real-time onboard deployment. In this paper, we present the first systematic review focused specifically on the integration of foundation models in mobile service robotics. We analyze how recent advances in foundation models address these core challenges through language-conditioned control, multimodal sensor fusion, uncertainty-aware reasoning, and efficient model scaling. We further examine real-world applications in domestic assistance, healthcare, and service automation, highlighting how foundation models enable context-aware, socially responsive, and generalizable robot behaviors. Beyond technical considerations, we discuss ethical, societal, and human-interaction implications associated with deploying foundation model-enabled service robots in human environments. Finally, we outline future research directions emphasizing reliability and lifelong adaptation, privacy-aware and resource-constrained deployment, and governance and human-in-the-loop frameworks required for safe, scalable, and trustworthy mobile service robotics.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 174 of its 317 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

174 of 317 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Mobile Robot Navigation Using Hand-Drawn Maps: A Vision Language Model Approach31 Jan
  2. 2024ReplanVLM: Replanning Robotic Tasks with Visual Language Models31 Jul
  3. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics15 Jun
  4. 2024VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Models30 Mar
  5. 2024Real-World Robot Applications of Foundation Models: A Review8 Feb
  6. 2023Physically Grounded Vision-Language Models for Robotic Manipulation5 Sep
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul
  8. 2023PaLM-E: An Embodied Multimodal Language Model6 Mar

Other references · 166

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2024The Llama 3 Herd of Modelscited by 14,192
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  9. 2023Segment Anythingcited by 8,158
  10. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  11. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  12. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
38 more
  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  6. 2024GPT-4o System Cardcited by 4,467
  7. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  9. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  10. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  12. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  13. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  14. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  15. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  16. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  17. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  18. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  19. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  20. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  21. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  22. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  23. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  24. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  25. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  26. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  27. 2025s1: Simple test-time scalingcited by 1,311
  28. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  29. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  30. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  31. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  32. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  33. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  34. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  35. 2020Mastering Atari with Discrete World Modelscited by 990
  36. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  37. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  38. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834

50 most cited of 166 All 166 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Foundation Model Driven Robotics: A Comprehensive Review
  2. 2026Foundation Models in Robotics: A Comprehensive Review of Methods, Models, Datasets, Challenges and Future Research Directions
  3. 2025Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interaction
  4. 2025From Grounding to Manipulation: Case Studies of Foundation Model Integration in Embodied Robotic Systems
  5. 2026The Embodiment Gap in Robot Foundation Models
  6. 2025Transforming Monolithic Foundation Models into Embodied Multi-Agent Architectures for Human-Robot Collaboration
  7. 2025How Good are Foundation Models in Step-by-Step Embodied Reasoning?
  8. 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectives