mlsanityData to 14 Oct 2026

Humanoid Agent via Embodied Chain-of-Action Reasoning with Multimodal Foundation Models for Zero-Shot Loco-Manipulation

arXiv 2504.09532 ↗First posted 13 Apr 2025cs.RO cs.AIcited by 3

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Humanoid loco-manipulation, which integrates whole-body locomotion with dexterous manipulation, remains a fundamental challenge in robotics. Beyond whole-body coordination and balance, a central difficulty lies in understanding human instructions and translating them into coherent sequences of embodied actions. Recent advances in foundation models provide transferable multimodal representations and reasoning capabilities, yet existing efforts remain largely restricted to either locomotion or manipulation in isolation, with limited applicability to humanoid settings. In this paper, we propose Humanoid-COA, the first humanoid agent framework that integrates foundation model reasoning with an Embodied Chain-of-Action (CoA) mechanism for zero-shot loco-manipulation. Within the perception--reasoning--action paradigm, our key contribution lies in the reasoning stage, where the proposed CoA mechanism decomposes high-level human instructions into structured sequences of locomotion and manipulation primitives through affordance analysis, spatial inference, and whole-body action reasoning. Extensive experiments on two humanoid robots, Unitree H1-2 and G1, in both an open test area and an apartment environment, demonstrate that our framework substantially outperforms prior baselines across manipulation, locomotion, and loco-manipulation tasks, achieving robust generalization to long-horizon and unstructured scenarios. Project page: https://humanoid-coa.github.io/

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 20 of its 26 references that are in the map. First posted on arXiv 13 Apr 2025.

Builds on

20 of 26 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents18 Jan

Other references · 19

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  3. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  4. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  5. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  6. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  7. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 203
  8. 2024HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulationcited by 112
  9. 2021Skill Induction and Planning with Latent Languagecited by 99
  10. 2023PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interactioncited by 76
  11. 2021Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real Worldcited by 71
  12. 2023Hierarchical Planning and Control for Box Loco-Manipulationcited by 44
7 more
  1. 2021Fully Autonomous Real-World Reinforcement Learning with Applications to Mobile Manipulationcited by 35
  2. 2023Grasp-Anything: Large-scale Grasp Dataset from Foundation Modelscited by 33
  3. 2024GAMap: Zero-Shot Object Goal Navigation with Multi-Scale Geometric-Affordance Guidancecited by 23
  4. 2024Zero-shot Object Navigation with Vision-Language Models Reasoningcited by 19
  5. 2024RS-MoE: A Vision-Language Model with Mixture of Experts for Remote Sensing Image Captioning and Visual Question Answeringcited by 15
  6. 2024Autonomous Behavior Planning For Humanoid Loco-manipulation Through Grounded Language Modelcited by 8
  7. 2024Socially-Aware Robot Navigation Enhanced by Bidirectional Natural Language Conversations Using Large Language Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Revisiting Embodied Chain-of-Thought for Generalizable Robot Manipulation
  2. 2024Grounding Language Models in Autonomous Loco-manipulation Tasks
  3. 2025Being-0: A Humanoid Robotic Agent with Vision-Language Models and Modular Skills
  4. 2026Guava: Distilling Frontier VLMs into a Compact Agent through a Robotic Manipulation Harness
  5. 2025Humanoid Locomotion and Manipulation: Current Progress and Challenges in Control, Planning, and Learning
  6. 2026ω-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulation
  7. 2026WholeBodyWAM: Generalizing Pre-trained World-Action Priors to Humanoid Loco-Manipulation via WBC-Grounded Coordination
  8. 2026Humanoid Loco-Manipulation With Discrete VLA Model