mlsanityData to 7 Oct 2026

EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action Models

arXiv 2605.25477 ↗First posted 25 May 2026cs.RO cs.AIcited by 7

On the map: Vision-language-action models → Reinforcement learning and post-training

Abstract (arXiv)

The ability to efficiently and reliably learn new tasks has been a foundational challenge in robotics. Vision-Language-Action (VLA) models have demonstrated strong generalization across diverse manipulation tasks, yet pretrained policies consistently fall short of the reliability required for real-world deployment. Reinforcement learning (RL) fine-tuning offers a promising path to bridge this gap, but existing approaches either train from scratch without fully leveraging pretrained priors, or fine-tune VLAs without achieving the sample efficiency and success rates that practical deployment demands. We present EXPO-FT, a system for stable, sample-efficient RL finetuning of pretrained VLA policies that closes this gap. Our system solves a suite of challenging manipulation tasks, including routing string lights and inserting the plug to light it up, striking a pool ball into a pocket, and inserting a flower into a wine bottle, each requiring combinations of high precision, dynamic actions, and robustness to varied initial states. Our system achieves perfect task performance (30/30 successes) across all evaluated tasks within an average of 19.1 minutes of online robot data, outperforming both prior RL-from-scratch and VLA finetuning approaches. We release an open-source codebase with the aim of facilitating broader adoption of RL finetuning of VLA models in robotics.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. Builds on 36 of its 47 references that are in the map. First posted on arXiv 25 May 2026.

Builds on

36 of 47 references.

Closest ancestors · 10

Same direction, newest first

  1. 2026RL Token: Bootstrapping Online RL with Vision-Language-Action Models24 Apr
  2. 2025π*0.6: a VLA That Learns From Experience18 Nov
  3. 2025Self-Improving Vision-Language-Action Models with Data Generation via Residual RL30 Oct
  4. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning11 Sep
  5. 2025Align-Then-stEer: Adapting the Vision-Language Action Models through Unified Latent Guidance2 Sep
  6. 2025What Can RL Bring to VLA Generalization? An Empirical Study26 May
  7. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning24 May
  8. 2025Interactive Post-Training for Vision-Language-Action Models22 May
  9. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy8 Feb
  10. 2025Improving Vision-Language-Action Model with Online Reinforcement Learning28 Jan

Other references · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  4. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  5. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  6. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  7. 2023Efficient Online Reinforcement Learning with Offline Datacited by 332
  8. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278
  9. 2024Diffusion Policy Policy Optimizationcited by 255
  10. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 245
  11. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learningcited by 153
  12. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
14 more
  1. 2023Learning a Diffusion Model Policy from Rewards via Q-Score Matchingcited by 126
  2. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learningcited by 105
  3. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous controlcited by 104
  4. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 71
  5. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 60
  6. 2025RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learningcited by 47
  7. 2025Residual Off-Policy RL for Finetuning Behavior Cloning Policiescited by 42
  8. 2026Q-learning with Adjoint Matchingcited by 33
  9. 2025EXPO: Stable Reinforcement Learning with Expressive Policiescited by 30
  10. 2023RLIF: Interactive Imitation Learning as Reinforcement Learningcited by 24
  11. 2025What Matters for Batch Online Reinforcement Learning in Robotics?cited by 14
  12. 2025Reinforcement Learning via Implicit Imitation Guidancecited by 12
  13. 2026TQL: Scaling Q-Functions with Transformers by Preventing Attention Collapsecited by 9
  14. 2026FASTER: Value-Guided Sampling for Fast RLcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Q-Learning With World Modelscited by 2
  2. 2026VLA-Precision: Asymmetric Co-Bootstrapping for Efficient Real-World Online RL of Vision-Language-Action Modelscited by 0
  3. 2026SCQ: Stabilizing Conservative Q-Learning with Sigmoid-Bounded Entropycited by 0
  4. 2026Reinforcement Learning for Real-Time Vision-Language-Action Policiescited by 0
  5. 2026From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Interventioncited by 0
  6. 2026Stable and Efficient Real-World Online VLA Post-Training via Asynchronous Replay-Anchored Policy Improvementcited by 0
  7. 2026Q-Learning with Scalar Adjoint Matchingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025π_textttRL: Online RL Fine-tuning for Flow-based Vision-Language-Action Models
  2. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
  3. 2025VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulators
  4. 2026ExToken: Structured Exploration for Efficient Vision-Language-Action Reinforcement Fine-tuning
  5. 2025Dual-Actor Fine-Tuning of VLA Models: A Talk-and-Tweak Human-in-the-Loop Approach
  6. 2026Z-1: Efficient Reinforcement Learning for Vision-Language-Action Models
  7. 2025EVOLVE-VLA: Test-Time Training from Environment Feedback for Vision-Language-Action Models
  8. 2026Colosseum V2: Benchmarking Generalization for Vision-Language-Action Models