mlsanityData to 7 Oct 2026

BEE: Intervention-Adaptive Real-World Reinforcement Learning with Vision-Language-Action Models

arXiv 2609.27450 ↗First posted 23 Sep 2026cs.RO cs.AIcited by 0

On the map: Vision-language-action models → Reinforcement learning and post-training

Abstract (arXiv)

Vision-language-action (VLA) models handle long-horizon manipulation, yet success hinges on a few precision-critical phases where millimeter-scale errors undo all prior progress. Online reinforcement learning (RL) can optimize exactly these actions, but free exploration is far too costly on real robots, which makes human corrections indispensable. However, existing online RL methods for VLAs either cannot incorporate such corrections or fold them into undifferentiated supervision. Yet human corrections are not uniformly noisy but reliable along some action dimensions and variable along others. Building on this, we introduce BEE, an intervention-adaptive framework for real-world RL on a frozen VLA that lets the policy go BEyond Expert imitation. We formulate human corrections not as actions to reproduce but as evidence about a constraint: a Correction Model predicts how a human would correct a given VLA proposal and how consistent the correction is along each action dimension. This predicted consistency sets the per-dimension tightness of a constraint on policy optimization. Where corrections are consistent the policy stays close to the human, and where they vary, the constraint relaxes. We evaluate BEE on three real-world manipulation tasks and one LIBERO-Pro simulation task at a matched online-data budget. BEE attains the highest success rate on every task, 91.2% on average against 57.5% for RLT and 42.1% for DSRL, and the lowest human intervention rate on all real-world tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. Builds on 37 of its 47 references that are in the map. First posted on arXiv 23 Sep 2026.

Builds on

37 of 47 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026RL Token: Bootstrapping Online RL with Vision-Language-Action Models24 Apr
  2. 2025π*0.6: a VLA That Learns From Experience18 Nov
  3. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning24 May
  4. 2025Interactive Post-Training for Vision-Language-Action Models22 May
  5. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy8 Feb
  6. 2025Improving Vision-Language-Action Model with Online Reinforcement Learning28 Jan
  7. 2024GRAPE: Generalizing Robot Policy via Preference Alignment28 Nov

Other references · 30

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  4. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  7. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  8. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  9. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  10. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  11. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  12. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
18 more
  1. 2018Reward Constrained Policy Optimizationcited by 397
  2. 2023Efficient Online Reinforcement Learning with Offline Datacited by 332
  3. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 245
  4. 2018Residual Reinforcement Learning for Robot Controlcited by 242
  5. 2025LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorizationcited by 166
  6. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learningcited by 153
  7. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
  8. 2018Residual Policy Learningcited by 146
  9. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 113
  10. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperationcited by 103
  11. 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 101
  12. 2021ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learningcited by 98
  13. 2024FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuningcited by 54
  14. 2025RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learningcited by 47
  15. 2024RLDG: Robotic Generalist Policy Distillation via Reinforcement Learningcited by 42
  16. 2023RLIF: Interactive Imitation Learning as Reinforcement Learningcited by 24
  17. 2025Learning from Active Human Involvement through Proxy Value Propagationcited by 16
  18. 2025Real-world Reinforcement Learning from Suboptimal Interventionscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Models
  2. 2026Agentic-VLA: Efficient Online Adaptation for Vision-Language-Action Models
  3. 2026SOP: A Scalable Online Post-Training System for Vision-Language-Action Models
  4. 2025EVOLVE-VLA: Test-Time Training from Environment Feedback for Vision-Language-Action Models
  5. 2026Prioritized Rollouts for Efficient World Model-based Vision-Language-Action Policy Optimization
  6. 2026WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RL
  7. 2026Vision-Language-Action Jump-Starting for Reinforcement Learning Robotic Agents
  8. 2026ROAD-VLA: Robust Online Adaptation via Self-Distillation for Vision-Language-Action Models