HELP: Human-Efficient Large-Scale Robot Post-Training with Rollout Segmentation
On the map: Vision-language-action models → Reinforcement learning and post-training
Abstract (arXiv)
When adapting Vision Language Action (VLA) models to downstream tasks, multiple rounds of post-training are often required to progressively address policy weaknesses. In this report, we focus on maximizing human efficiency during this iterative process, measured by policy improvement and task throughput per unit of human labor and time. We propose HELP, a Human-Efficient Large-scale robot Post-training pipeline in which two specialized operators supervise twelve robots concurrently. A trained Teleoperator provides high-value remote interventions and recovery demonstrations, while a Floor Operator monitors the robot fleet, triggers takeovers, and performs physical resets. This role specialization improves human efficiency by reducing task switching, lowering operator training costs, and expanding robot interaction coverage. Beyond increasing rollout volume, concurrent supervision also broadens the range of policy behaviors observed by the human team, making recurring failure modes easier to identify and enabling more targeted takeovers, resets, and recovery demonstrations. To efficiently utilize the large and mixed-quality rollout data, HELP incorporates vlac, an automatic rollout segmentation critic specifically designed for this setting. It separates autonomous trajectories into progress-making, idle, failure-inducing, and recovery segments. Useful rollout segments are retained and combined with Human-in-the-Loop data for the next post-training round. Across four real-world manipulation tasks, HELP achieves 80%--95% success rates and improves task throughput by 1.7×--4.2× over the base model. Under matched HITL recovery budgets, VLAC-CUT further amplifies throughput gains by 1.20×--3.43× and success-rate gains by 1.50×--3.00× over HITL-only updates.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. Builds on 71 of its 76 references that are in the map. First posted on arXiv 8 Jul 2026.
Builds on
71 of 76 references.
Closest ancestors · 6
Same direction, newest first
- 2026Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policies1 May
- 2026AtomVLA: Scalable Post-Training for Robotic Manipulation via Predictive Latent World Models9 Mar
- 2025π*0.6: a VLA That Learns From Experience18 Nov
- 2025World-Env: Leveraging World Model as a Virtual Environment for VLA Post-Training29 Sep
- 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learning19 Sep
- 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy8 Feb
Other references · 65
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Visual Instruction Tuningcited by 7,792
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2017Hindsight Experience Replaycited by 1,096
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
- 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
- 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
38 more
- 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
- 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
- 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
- 2019Learning Latent Plans from Playcited by 310
- 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 180
- 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 171
- 2025Flow Q-Learningcited by 162
- 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 155
- 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learningcited by 153
- 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackcited by 147
- 2023Vision-Language Models as Success Detectorscited by 138
- 2025RewardBench 2: Advancing Reward Model Evaluationcited by 136
- 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policiescited by 128
- 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
- 2023RoboVQA: Multimodal Long-Horizon Reasoning for Roboticscited by 116
- 2025Reinforcement Learning with Action Chunkingcited by 97
- 2023Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learningcited by 93
- 2024VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Modelscited by 85
- 2024Vision Language Models are In-Context Value Learnerscited by 84
- 2023Video Prediction Models as Rewards for Reinforcement Learningcited by 76
- 2016Unsupervised Perceptual Rewards for Imitation Learningcited by 64
- 2024Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Datacited by 58
- 2025ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrationscited by 52
- 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 51
- 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 49
- 2026RoboReward: General-Purpose Vision-Language Reward Models for Roboticscited by 46
- 2025Multimodal RewardBench: Holistic Evaluation of Reward Models for Vision Language Modelscited by 40
- 2025RoboFAC: A Comprehensive Framework for Robotic Failure Analysis and Correctioncited by 37
- 2025Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulationcited by 37
- 2025SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulationcited by 36
- 2025MMSI-Video-Bench: A Holistic Benchmark for Video-Based Spatial Intelligencecited by 34
- 2025Flow-Based Policy for Online Reinforcement Learningcited by 33
- 2024Autonomous Improvement of Instruction Following Skills via Foundation Modelscited by 32
- 2026TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Roboticscited by 29
- 2025Self-Improving Embodied Foundation Modelscited by 28
- 2024Practice Makes Perfect: Planning to Learn Skill Parameter Policiescited by 22
- 2023Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotioncited by 21
- 2024Real-World Offline Reinforcement Learning from Vision Language Model Feedbackcited by 20
50 most cited of 65 All 65 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Training
- 2026SOP: A Scalable Online Post-Training System for Vision-Language-Action Models
- 2026PhAIL: A Real-Robot VLA Benchmark and Distributional Methodology
- 2026Temporal GRPO: Beyond Trajectory-Level Credit in Vision-Language-Action Reinforcement Learning
- 2026Kintsugi-VLA: Turning Failed Robot Rollouts into Recovery Data through Interventional Recoverability
- 2026Prism-GRPO: Faster VLA Policy Optimization via Splitting Same-outcome Groups
- 2026Prioritized Rollouts for Efficient World Model-based Vision-Language-Action Policy Optimization
- 2023SPRINT: Scalable Policy Pre-Training via Language Instruction Relabeling