mlsanityData to 7 Oct 2026

What Language Does and What the Evidence Supports: A Functional Role Taxonomy and Evidence Audit of Language Grounding in Embodied Agents

arXiv 2608.03099 ↗First posted 4 Aug 2026cs.CLcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Foundation models place language throughout embodied agents, but its presence does not show what it contributes or how well that contribution is grounded. This survey separates these two questions. We define five non-exclusive functional roles for language: Specification, Embodied Representation, Action Orchestration, Grounding Regulation, and Execution Coupling. For each role, we trace the path from linguistic content to its embodied consumer and identify the observations or interventions that can test the claimed responsibility. Applying this framework to the reviewed literature reveals a recurring gap between functional use and evidential support. Interpretable or revised linguistic intermediates may be incorrect, go unused, or fail to affect later behavior. Even when actions are directly conditioned on language, system-level success does not by itself isolate language's contribution. We therefore evaluate grounding claim by claim, asking whether the reported evidence supports the specific responsibility assigned to language. Using role claims rather than architectures as the unit of comparison allows us to compare modular and end-to-end embodied agents without extending conclusions beyond the reported evidence.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 72 of its 98 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

72 of 98 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents3 Feb

Other references · 71

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  3. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  4. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  5. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  6. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  7. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  8. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  9. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
  10. 2022Visual Language Maps for Robot Navigationcited by 362
  11. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  12. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
38 more
  1. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  2. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  3. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 276
  4. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  5. 2020Experience Grounds Languagecited by 241
  6. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 191
  7. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 187
  8. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 170
  9. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  10. 2023Physically Grounded Vision-Language Models for Robotic Manipulationcited by 163
  11. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 157
  12. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 153
  13. 2025ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planningcited by 152
  14. 2023Vision-Language Models as Success Detectorscited by 138
  15. 2023Distilled Feature Fields Enable Few-Shot Language-Guided Manipulationcited by 130
  16. 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 126
  17. 2024AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulationcited by 118
  18. 2025ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Modelcited by 112
  19. 2024GRAPE: Generalizing Robot Policy via Preference Alignmentcited by 106
  20. 2025OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoningcited by 105
  21. 2025UP-VLA: A Unified Understanding and Prediction Model for Embodied Agentcited by 99
  22. 2024CoPa: General Robotic Manipulation through Spatial Constraints of Parts with Foundation Modelscited by 97
  23. 2023Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modellingcited by 88
  24. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 87
  25. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 77
  26. 2023GPT-4V(ision) for Robotics: Multimodal Task Planning from Human Demonstrationcited by 76
  27. 2023Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigationcited by 74
  28. 2024Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphscited by 72
  29. 2025CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsificationcited by 70
  30. 2025MemER: Scaling Up Memory for Robot Control via Experience Retrievalcited by 64
  31. 2023A2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Modelscited by 61
  32. 2025EnerVerse: Envisioning Embodied Future Space for Robotics Manipulationcited by 59
  33. 2025OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraintscited by 58
  34. 2025VTLA: Vision-Tactile-Language-Action Model with Preference Learning for Insertion Manipulationcited by 54
  35. 2025RoboGround: Robotic Manipulation with Grounded Vision-Language Priorscited by 50
  36. 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 49
  37. 2025ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledgecited by 49
  38. 2023Towards A Unified Agent with Foundation Modelscited by 46

50 most cited of 71 All 71 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Synthetic Linguistic Agency: How an Embodied Mortal Agent Learns Linguistic Affordances through Consequential Social Experience
  2. 2025Ground-Compose-Reinforce: Grounding Language in Agentic Behaviours using Limited Data
  3. 2026From Cognitive Architectures to Language Agents: A Mechanism-Level Review of Lineage, Convergence, and Migration Gaps
  4. 2024LangSuitE: Planning, Controlling and Interacting with Large Language Models in Embodied Text Environments
  5. 2026Seeing Isn't Believing: Mitigating Belief Inertia via Active Intervention in Embodied Agents
  6. 2024Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making
  7. 2026Do Vision-Language-Action Models Mean What They Say? On the Role of Faithfulness in Embodied Reasoning
  8. 2024"What's my model inside of?": Exploring the role of environments for grounded natural language understanding