# Z-1: Efficient Reinforcement Learning for Vision-Language-Action Models

arXiv: https://arxiv.org/abs/2606.31846 · HTML: https://mlsanity.com/paper/2606.31846/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models.
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 30 Jun 2026.

## Placed in

- Vision-language-action models / [Reinforcement learning and post-training](https://mlsanity.com/vla/reinforcement-learning-and-post-training/)
- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 18 of 18)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [Behavioral Cloning from Observation](https://mlsanity.com/paper/1805.01954/) (2018)
- [RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots](https://mlsanity.com/paper/2406.02523/) (2024)
- [π*0.6: a VLA That Learns From Experience](https://mlsanity.com/paper/2511.14759/) (2025)
- [VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning](https://mlsanity.com/paper/2505.18719/) (2025)
- [π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities](https://mlsanity.com/paper/2604.15483/) (2026)
- [ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learning](https://mlsanity.com/paper/2505.22094/) (2025)
- [Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising](https://mlsanity.com/paper/2604.26694/) (2026)
- [DepthVLA: Enhancing Vision-Language-Action Models with Depth-Aware Spatial Reasoning](https://mlsanity.com/paper/2510.13375/) (2025)
- [VLA-R1: Enhancing Reasoning in Vision-Language-Action Models](https://mlsanity.com/paper/2510.01623/) (2025)
- [Robot Data Curation with Mutual Information Estimators](https://mlsanity.com/paper/2502.08623/) (2025)
- [Don't Blind Your VLA: Aligning Visual Representations for OOD Generalization](https://mlsanity.com/paper/2510.25616/) (2025)
- [SOP: A Scalable Online Post-Training System for Vision-Language-Action Models](https://mlsanity.com/paper/2601.03044/) (2026)
- [TreeAdv: Tree-Structured Advantage Redistribution for Group-Based RL](https://mlsanity.com/paper/2601.03703/) (2026)

## Built on by (most cited first, 2 of 2)

- [Prism-GRPO: Faster VLA Policy Optimization via Splitting Same-outcome Groups](https://mlsanity.com/paper/2608.17423/) (2026)
- [Train Together or Merge Later? Unifying VLA Experts via a Shared Action Interface](https://mlsanity.com/paper/2609.33125/) (2026)

## Abstract (arXiv)

Vision-Language-Action (VLA) models offer a promising framework for robotic manipulation by connecting language instructions, visual observations, and continuous control. However, most existing policies remain limited by behavior cloning or supervised fine-tuning (SFT) from fixed demonstrations, which provides limited opportunity to improve from the policy's own failures. In this paper, we present Z-1, a reinforcement learning (RL) post-training framework for flow-based VLA models. Built on top of π0.5, Z-1 uses only publicly released RoboCasa demonstrations for SFT and then applies a task-wise Group Relative Policy Optimization (GRPO) strategy across 24 standard RoboCasa tasks. To improve the efficiency and stability of online optimization, Z-1 combines shared-prefix rollout construction, tree-structured trajectory branching, completion-aware reward calibration, and selective joint training of VLM and Action Expert. Across all 24 RoboCasa tasks, Z-1 achieves an average success rate of 80.6%, improving over its SFT initialization by 13.2% points and outperforms the published sota models. These results show that systematic GRPO post-training can substantially improve flow-based VLA policies without additional private demonstrations.