# Cybo-Waiter: A Physical Agentic Framework for Humanoid Whole-Body Locomotion-Manipulation

arXiv: https://arxiv.org/abs/2603.10675 · HTML: https://mlsanity.com/paper/2603.10675/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 28 of its 30 references that are in the map.
- First posted on arXiv 11 Mar 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 28 of 28)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Inner Monologue: Embodied Reasoning through Planning with Language Models](https://mlsanity.com/paper/2207.05608/) (2022)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models](https://mlsanity.com/paper/2307.05973/) (2023)
- [CLIPort: What and Where Pathways for Robotic Manipulation](https://mlsanity.com/paper/2109.12098/) (2021)
- [Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation](https://mlsanity.com/paper/2209.05451/) (2022)
- [Real-World Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2303.03381/) (2023)
- [BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion](https://mlsanity.com/paper/2508.08241/) (2025)
- [Generalizable Humanoid Manipulation with 3D Diffusion Policies](https://mlsanity.com/paper/2410.10803/) (2024)
- [Chat with the Environment: Interactive Multimodal Perception Using Large Language Models](https://mlsanity.com/paper/2303.08268/) (2023)
- [LeVERB: Humanoid Whole-Body Control with Latent Vision-Language Instruction](https://mlsanity.com/paper/2506.13751/) (2025)
- [WholeBodyVLA: Towards Unified Latent VLA for Whole-Body Loco-Manipulation Control](https://mlsanity.com/paper/2512.11047/) (2025)
- [ReplanVLM: Replanning Robotic Tasks with Visual Language Models](https://mlsanity.com/paper/2407.21762/) (2024)
- [Being-0: A Humanoid Robotic Agent with Vision-Language Models and Modular Skills](https://mlsanity.com/paper/2503.12533/) (2025)
- [VerifyLLM: LLM-Based Pre-Execution Task Plan Verification for Robots](https://mlsanity.com/paper/2507.05118/) (2025)
- [Autonomous Behavior Planning For Humanoid Loco-manipulation Through Grounded Language Model](https://mlsanity.com/paper/2408.08282/) (2024)
- [PLanAR: Planning-Language-Grounded Agentic Reasoning for Robot Manipulation](https://mlsanity.com/paper/2602.01662/) (2026)
- [Hierarchical Vision-Language Planning for Multi-Step Humanoid Manipulation](https://mlsanity.com/paper/2506.22827/) (2025)
- [Scene Graph-Guided Proactive Replanning for Failure-Resilient Embodied Agent](https://mlsanity.com/paper/2508.11286/) (2025)
- [iWalker: Imperative Visual Planning for Walking Humanoid Robot](https://mlsanity.com/paper/2409.18361/) (2024)

## Built on by (most cited first, 2 of 2)

- [FRAMES: Failure Recovery And Monitoring of Embodied Skills for Humanoid Loco-Manipulation](https://mlsanity.com/paper/2609.22538/) (2026)
- [InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-Manipulation](https://mlsanity.com/paper/2610.02196/) (2026)

## Abstract (arXiv)

Robots are increasingly expected to execute open ended natural language requests in human environments, which demands reliable long horizon execution under partial observability. This is especially challenging for humanoids because locomotion and manipulation are tightly coupled through stance, reachability, and balance. We present a humanoid agent framework that turns VLM plans into verifiable task programs and closes the loop with multi object 3D geometric supervision. A VLM planner compiles each instruction into a typed JSON sequence of subtasks with explicit predicate based preconditions and success conditions. Using SAM3 and RGB-D, we ground all task relevant entities in 3D, estimate object centroids and extents, and evaluate predicates over stable frames to obtain condition level diagnostics. The supervisor uses these diagnostics to verify subtask completion and to provide condition-level feedback for progression and replanning. We execute each subtask by coordinating humanoid locomotion and whole-body manipulation, selecting feasible motion primitives under reachability and balance constraints. Experiments on tabletop manipulation and long horizon humanoid loco manipulation tasks show improved robustness from multi object grounding, temporal stability, and recovery driven replanning.