# VLAgents: A Policy Server for Efficient VLA Inference

arXiv: https://arxiv.org/abs/2601.11250 · HTML: https://mlsanity.com/paper/2601.11250/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Efficient VLA inference and adaptation, one of 13 directions in Vision-language-action models.
- Builds on 13 of its 16 references that are in the map.
- First posted on arXiv 16 Jan 2026.

## Placed in

- Vision-language-action models / [Efficient VLA inference and adaptation](https://mlsanity.com/vla/efficient-vla-inference-and-adaptation/)

## Builds on (most cited first, 13 of 13)

- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [FAST: Efficient Action Tokenization for Vision-Language-Action Models](https://mlsanity.com/paper/2501.09747/) (2025)
- [Evaluating Real-World Robot Manipulation Policies in Simulation](https://mlsanity.com/paper/2405.05941/) (2024)
- [ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI](https://mlsanity.com/paper/2410.00425/) (2024)
- [Refined Policy Distillation: From VLA Generalists to RL Experts](https://mlsanity.com/paper/2503.05833/) (2025)
- [Augmented Reality for RObots (ARRO): Pointing Visuomotor Policies Towards Visual Robustness](https://mlsanity.com/paper/2505.08627/) (2025)
- [Robot Control Stack: A Lean Ecosystem for Robot Learning at Scale](https://mlsanity.com/paper/2509.14932/) (2025)

## Built on by (most cited first, 3 of 3)

- [vla.cpp: A Unified Inference Runtime for Vision-Language-Action Models](https://mlsanity.com/paper/2606.08094/) (2026)
- [DuoBench: A Reproducible Benchmark for Bimanual Manipulation in Simulation and the Real World](https://mlsanity.com/paper/2606.11901/) (2026)
- [DUET-DINO: Simultaneous Cross-View World Modeling for Latent Planning in Robot Manipulation](https://mlsanity.com/paper/2609.10506/) (2026)

## Abstract (arXiv)

The rapid emergence of Vision-Language-Action models (VLAs) has a significant impact on robotics. However, their deployment remains complex due to the fragmented interfaces and the inherent communication latency in distributed setups. To address this, we introduce VLAgents, a modular policy server that abstracts VLA inferencing behind a unified Gymnasium-style protocol. Crucially, its communication layer transparently adapts to the context by supporting both zero-copy shared memory for high-speed simulation and compressed streaming for remote hardware. In this work, we present the architecture of VLAgents and validate it by integrating seven policies -- including OpenVLA and Pi Zero. In a benchmark with both local and remote communication, we further demonstrate how it outperforms the default policy servers provided by OpenVLA, OpenPi, and LeRobot. VLAgents is available at https://github.com/RobotControlStack/vlagents