mlsanityData to 14 Oct 2026

Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interaction

arXiv 2508.05294 ↗First posted 7 Aug 2025cs.RO cs.AI cs.LGcited by 19

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Foundation models, including large language models (LLMs) and vision-language models (VLMs), have recently enabled novel approaches to robot autonomy and human-robot interfaces. In parallel, vision-language-action models (VLAs) or large behavior models (LBMs) are increasing the dexterity and capabilities of robotic systems. This survey paper reviews works that advance agentic applications and architectures, including initial efforts with GPT-style interfaces and more complex systems where AI agents function as coordinators, planners, perception actors, or generalist interfaces. Such agentic architectures allow robots to reason over natural language instructions, invoke APIs, plan task sequences, or assist in operations and diagnostics. In addition to peer-reviewed research, due to the fast-evolving nature of the field, we highlight and include community-driven projects, ROS packages, and industrial frameworks that show emerging trends. We propose a taxonomy for classifying model integration approaches and present a comparative analysis of the role that agents play in different solutions in today's literature.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 47 of its 66 references that are in the map. First posted on arXiv 7 Aug 2025.

Builds on

47 of 66 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024BUMBLE: Unifying Reasoning and Acting with Vision-Language Models for Building-wide Mobile Manipulation8 Oct
  2. 2024Enhancing Robot Explanation Capabilities through Vision-Language Models: a Preliminary Study by Interpreting Visual Inputs for Improved Human-Robot Interaction15 Apr
  3. 2024Real-World Robot Applications of Foundation Models: A Review8 Feb
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 43

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  3. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  5. 2018World Modelscited by 1,513
  6. 2023Mastering Diverse Domains through World Modelscited by 1,253
  7. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  8. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  9. 2016End to End Learning for Self-Driving Carscited by 1,009
  10. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  11. 2020Reinforcement Learning with Augmented Datacited by 468
  12. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
31 more
  1. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
  2. 2023An Embodied Generalist Agent in 3D Worldcited by 365
  3. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 299
  4. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  5. 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 216
  6. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 169
  7. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 169
  8. 2023SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Modelscited by 153
  9. 2023Large Language Models for Robotics: A Surveycited by 133
  10. 2023RoboCat: A Self-Improving Generalist Agent for Robotic Manipulationcited by 111
  11. 2025A survey of agent interoperability protocols: Model Context Protocol (MCP), Agent Communication Protocol (ACP), Agent-to-Agent Protocol (A2A), and Agent Network Protocol (ANP)cited by 101
  12. 2024AIOS: LLM Agent Operating Systemcited by 97
  13. 2025M-LLM Based Video Frame Selection for Efficient Video Understandingcited by 88
  14. 2024AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agentscited by 87
  15. 2023Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigationcited by 74
  16. 2025ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledgecited by 49
  17. 2023Robot Learning in the Era of Foundation Models: A Surveycited by 38
  18. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  19. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  20. 2025MoManipVLA: Transferring Vision-language-action Models for General Mobile Manipulationcited by 34
  21. 2024LaMMA-P: Generalizable Multi-Agent Long-Horizon Task Allocation and Planning with LM-Driven PDDL Plannercited by 32
  22. 2024Odyssey: Empowering Minecraft Agents with Open-World Skillscited by 26
  23. 2024SELP: Generating Safe and Efficient Task Plans for Robot Agents with Large Language Modelscited by 21
  24. 2021Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligencecited by 18
  25. 2025LLM-Empowered Embodied Agent for Memory-Augmented Task Planning in Household Roboticscited by 17
  26. 2024Enabling Novel Mission Operations and Interactions with ROSA: The Robot Operating System Agentcited by 16
  27. 2024Large Language Models for Orchestrating Bimanual Robotscited by 10
  28. 2024ConceptAgent: LLM-Driven Precondition Grounding and Tree Search for Robust Task Planning and Executioncited by 8
  29. 2025RAI: Flexible Agent Framework for Embodied AIcited by 5
  30. 2025Distributed AI Agents for Cognitive Underwater Robot Autonomycited by 4
  31. 2025General-Purpose Robotic Navigation via LVLM-Orchestrated Perception, Reasoning, and Actingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 14.

19 papers.

Sort:

By area:

  1. 2026Spark: Strategic Policy-Aware Exploration via Dynamic Branching for Long-Horizon Agentic Learningcited by 11
  2. 2026Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Modelscited by 9
  3. 2026Agentic AI Empowered Intent-Based Networking for 6Gcited by 7
  4. 2025VL4Gaze: Unleashing Vision-Language Models for Gaze Followingcited by 3
  5. 2026From Prompt to Physical Actuation: Holistic Threat Modeling of LLM-Enabled Robotic Systemscited by 3
  6. 2025Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmarkcited by 2
  7. 2026You Don't Need To Stay in The Loop: An Agentic Robotics Loop for Robot-Policy Improvementcited by 2
  8. 2025Agentic Scene Policies: Unifying Space, Semantics, and Affordances for Robot Actioncited by 1
  9. 2025ROSBag MCP Server: Analyzing Robot Data with LLMs for Agentic Embodied AI Applicationscited by 1
  10. 2025Leveraging Adaptive Group Negotiation for Heterogeneous Multi-Robot Collaboration with Large Language Modelscited by 1
  11. 2026A Framework for Low-Latency, LLM-driven Multimodal Interaction on the Pepper Robotcited by 1
  12. 2026Agentic AI for Self-Driving Laboratories in Soft Matter: Taxonomy, Benchmarks,and Open Challengescited by 0
7 more
  1. 2026Adversarial Moral Stress Testing of Large Language Modelscited by 0
  2. 2026ReXSonoVQA: A Video QA Benchmark for Procedure-Centric Ultrasound Understandingcited by 0
  3. 2026Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligencecited by 0
  4. 2026Toward Unified Robot Learning: Bridging Representation, Vision-Language-Action, and World Modelscited by 0
  5. 2026Finder: Agentic Closed-Loop Object Finding for Embodied Groundingcited by 0
  6. 2026OverForge: Reasoning Through Strategies and Tactics Helps Cooperative Lifelong Adaptationcited by 0
  7. 2026Words to World: Benchmarking LLMs in Interactive Physical Environmentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey on Vision-Language-Action Models for Embodied AI
  2. 2025PFEA: An LLM-based High-Level Natural Language Planning and Feedback Embodied Agent for Human-Centered AI
  3. 2023The Rise and Potential of Large Language Model Based Agents: A Survey
  4. 2025Fundamentals of Building Autonomous LLM Agents
  5. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
  6. 2025Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challenges
  7. 2026From Language to Action: Can LLM-Based Agents Be Used for Embodied Robot Cognition?
  8. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey