Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Models
On the map: Vision-language-action models → Generalist VLA models and benchmarks
also in 1 more
Abstract (arXiv)
Vision-language-action (VLA) models are effective robot action executors, but they remain limited on long-horizon tasks due to the dual burden of extended closed-loop planning and diverse physical operations. We therefore propose VLAs-as-Tools, a strategy that distributes this burden across a high-level vision language model (VLM) agent for temporal reasoning and a family of specialized VLA tools for diverse local physical operations. The VLM handles scene analysis, global planning, and recovery, while each VLA tool executes a bounded subtask. To tightly couple agent planning with VLA tool execution in long-horizon tasks, we introduce a VLA tool-family interface that exposes explicit tool selection and in-execution progress feedback, enabling efficient event-triggered agent replanning without continuous agent polling. To obtain diverse specialized VLA tools that faithfully follow agent invocations, we further propose Tool-Aligned Post-Training (TAPT), which constructs invocation-aligned training units for instruction following and adopts tool-family residual adapters for efficient tool specialization. Experiments show that VLAs-as-Tools improves the success rate of π0.5 by 4.8 points on LIBERO-Long and 23.1 points on RoboTwin, and further enhances invocation fidelity by 15.0 points as measured by Non-biased Rate. Code will be released.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models. Builds on 41 of its 48 references that are in the map. First posted on arXiv 13 May 2026.
Builds on
41 of 48 references.
Closest ancestors · 9
Same direction, newest first
- 2025Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgetting26 Sep
- 2025Long-VLA: Unleashing Long-Horizon Capability of Vision Language Action Model for Robot Manipulation27 Aug
- 2025LoHoVLA: A Unified Vision-Language-Action Model for Long-Horizon Embodied Tasks31 May
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalization22 Apr
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success27 Feb
- 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control9 Feb
- 2024π0: A Vision-Language-Action Flow Model for General Robot Control31 Oct
- 2024OpenVLA: An Open-Source Vision-Language-Action Model13 Jun
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul
Other references · 32
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
- 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
20 more
- 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
- 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
- 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Modelscited by 278
- 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 216
- 2024RT-H: Action Hierarchies Using Languagecited by 204
- 2025ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planningcited by 152
- 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 137
- 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 117
- 2025Robo2VLM: Visual Question Answering from Large-Scale In-the-Wild Robot Manipulation Datasetscited by 39
- 2025Agentic Robot: A Brain-Inspired Framework for Vision-Language-Action Models in Embodied Agentscited by 30
- 2026When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAscited by 27
- 2026AtomVLA: Scalable Post-Training for Robotic Manipulation via Predictive Latent World Modelscited by 21
- 2026PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulationcited by 20
- 2025Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interactioncited by 18
- 2026AtomicVLA: Unlocking the Potential of Atomic Skill Learning in Robotscited by 15
- 2026LiLo-VLA: Compositional Long-Horizon Manipulation via Linked Object-Centric Policiescited by 12
- 2026Towards Long-Lived Robots: Continual Learning VLA Models via Reinforcement Fine-Tuningcited by 11
- 2026Restoring Linguistic Grounding in VLA Models via Train-Free Attention Recalibrationcited by 11
- 2018Action Representations in Robotics: A Taxonomy and Systematic Classificationcited by 9
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Modelscited by 47
- 2026Don't Drop the BATON: Long-Horizon Robot Manipulation via Agentic Subtask Exploration and Transition-aware Memorycited by 4
- 2026SpatialCLI: Learning to Reason With Spatial Tools, Then Without Themcited by 1
- 2026Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigationcited by 1
- 2026Ludiscriptscriptstyle 0.1: An Agentic System for Socially Intelligent Robotscited by 0
- 2026DualManip: Agentic Dynamic Manipulation via Dual-Path Semantic Reasoning and Geometric Adaptationcited by 0
- 2026CogWAM: Aligning Semantic Cognition with World Action Modeling via Event-Driven Interfacescited by 0
- 2026RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomescited by 0
- 2026RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomes8 Oct 2026
- 2026CogWAM: Aligning Semantic Cognition with World Action Modeling via Event-Driven Interfaces29 Sep 2026
- 2026DualManip: Agentic Dynamic Manipulation via Dual-Path Semantic Reasoning and Geometric Adaptation25 Sep 2026
- 2026Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigation31 Aug 2026
- 2026Ludiscriptscriptstyle 0.1: An Agentic System for Socially Intelligent Robots22 Aug 2026
- 2026Don't Drop the BATON: Long-Horizon Robot Manipulation via Agentic Subtask Exploration and Transition-aware Memory17 Aug 2026
- 2026SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them30 Jul 2026
- 2026Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models16 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents
- 2026CoRE-VLA: Towards Scalable and Robust Vision-Language-Action Modeling via Conditional Routing of Experts
- 2026Explicit Language Memory for Long-Horizon Planning in Vision-Language-Action Models
- 2024A Survey on Vision-Language-Action Models for Embodied AI
- 2026ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models
- 2026Evolve Vision-Language-Action Model into an Agent with On-the-fly Tool-use
- 2026Rethinking the Practicality of Vision-language-action Model: A Comprehensive Benchmark and An Improved Baseline
- 2026Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents