# VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation

arXiv: https://arxiv.org/abs/2606.07723 · HTML: https://mlsanity.com/paper/2606.07723/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2026.

## Built on by (most cited first, 11 of 11)

- [Show-Harness: Just a VLM Agent Can Play Robots](https://mlsanity.com/paper/2609.10522/) (2026)
- [VIA: Visual Interface Agent for Robot Control](https://mlsanity.com/paper/2607.11119/) (2026)
- [Don't Drop the BATON: Long-Horizon Robot Manipulation via Agentic Subtask Exploration and Transition-aware Memory](https://mlsanity.com/paper/2608.16889/) (2026)
- [SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them](https://mlsanity.com/paper/2607.27703/) (2026)
- [Recursive Harness Distillation across Agents for Robot Manipulation](https://mlsanity.com/paper/2609.33378/) (2026)
- [In-Context Learning for Robots: Methods and Applications](https://mlsanity.com/paper/2609.36012/) (2026)
- [Skills in Weights, Memory in Code: Hybrid Learning for Memory-Dependent Robot Manipulation](https://mlsanity.com/paper/2608.09410/) (2026)
- [MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulation](https://mlsanity.com/paper/2609.38078/) (2026)
- [Beyond the Current Scene: Event-Referential Grasping with Active View Selection](https://mlsanity.com/paper/2609.39375/) (2026)
- [RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomes](https://mlsanity.com/paper/2610.11480/) (2026)
- [LADDER: A Long-Horizon Robot Manipulation Benchmark](https://mlsanity.com/paper/2610.18053/) (2026)

## Abstract (arXiv)

Open-vocabulary long-horizon manipulation requires robots to reason over flexible instructions and complex multi-object scenes while adaptively planning, executing, monitoring, and recovering from failures. We address these demands with a closed agent loop in which a VLM orchestrates heterogeneous robot capabilities as interruptible tools. Unlike in virtual AI agents, the timing of decisions, actions and tool calls is important in a physical world that does not pause for reasoning. We refer to this setting as Physical Orchestration, and propose VoLoAgent, a VLM that plans, monitors, and recovers by treating a VLA/WAM as an interruptible tool it steers mid-rollout alongside vision models and action primitives. To evaluate these long-horizon capabilities, we introduce RoboVoLo, a high-fidelity benchmark for open-vocabulary long-horizon manipulation across common sense, memory/state tracking, complex references, and world knowledge, with both task-level success and failure-mode diagnostics. Experiments show VoLoAgent substantially outperforms single VLA/VLM or tool-based systems, with validation on real-robot experiments. Project page: https://chicychen.github.io/VoLo/