mlsanityData to 7 Oct 2026

SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning

arXiv 2509.09674 ↗First posted 11 Sep 2025cs.RO cs.AI cs.CLcited by 141

On the map: Vision-language-action models → Reinforcement learning and post-training

Abstract (arXiv)

Vision-Language-Action (VLA) models have recently emerged as a powerful paradigm for robotic manipulation. Despite substantial progress enabled by large-scale pretraining and supervised fine-tuning (SFT), these models face two fundamental challenges: (i) the scarcity and high cost of large-scale human-operated robotic trajectories required for SFT scaling, and (ii) limited generalization to tasks involving distribution shift. Recent breakthroughs in Large Reasoning Models (LRMs) demonstrate that reinforcement learning (RL) can dramatically enhance step-by-step reasoning capabilities, raising a natural question: Can RL similarly improve the long-horizon step-by-step action planning of VLA? In this work, we introduce SimpleVLA-RL, an efficient RL framework tailored for VLA models. Building upon veRL, we introduce VLA-specific trajectory sampling, scalable parallelization, multi-environment rendering, and optimized loss computation. When applied to OpenVLA-OFT, SimpleVLA-RL achieves SoTA performance on LIBERO and even outperforms π0 on RoboTwin 1.0&2.0 with the exploration-enhancing strategies we introduce. SimpleVLA-RL not only reduces dependence on large-scale data and enables robust generalization, but also remarkably surpasses SFT in real-world tasks. Moreover, we identify a novel phenomenon ``pushcut'' during RL training, wherein the policy discovers previously unseen patterns beyond those seen in the previous training process. Github: https://github.com/PRIME-RL/SimpleVLA-RL

Key facts. Cited by 141 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. Builds on 68 of its 75 references that are in the map. First posted on arXiv 11 Sep 2025.

Builds on

68 of 75 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025TGRPO :Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimization10 Jun
  2. 2025What Can RL Bring to VLA Generalization? An Empirical Study26 May
  3. 2025RFTF: Reinforcement Fine-tuning for Embodied Agents with Temporal Feedback26 May
  4. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning24 May
  5. 2025Interactive Post-Training for Vision-Language-Action Models22 May
  6. 2025ReinboT: Amplifying Robot Visual-Language Manipulation with Reinforcement Learning12 May
  7. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy8 Feb
  8. 2025Improving Vision-Language-Action Model with Online Reinforcement Learning28 Jan
  9. 2024GRAPE: Generalizing Robot Policy via Preference Alignment28 Nov
  10. 2024Octo: An Open-Source Generalist Robot Policy20 May

Other references · 58

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  6. 2025Qwen3 Technical Reportcited by 8,195
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  9. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  10. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  11. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  12. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
38 more
  1. 2015Trust Region Policy Optimizationcited by 2,503
  2. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  3. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  4. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  5. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  6. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  7. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  8. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  9. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  10. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  11. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  12. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  13. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
  14. 2025Process Reinforcement through Implicit Rewardscited by 478
  15. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
  16. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 463
  17. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  18. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  19. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  20. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 340
  21. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
  22. 2025TTRL: Test-Time Reinforcement Learningcited by 253
  23. 2017Inverse Reward Designcited by 246
  24. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 236
  25. 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 212
  26. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  27. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Modelscited by 187
  28. 2024The Effect of Sampling Temperature on Problem Solving in Large Language Modelscited by 168
  29. 2019A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithmscited by 136
  30. 2025NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Taskscited by 119
  31. 2025GR-3 Technical Reportcited by 110
  32. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 104
  33. 2021Reward (Mis)design for Autonomous Drivingcited by 102
  34. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  35. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learningcited by 88
  36. 2025AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergycited by 75
  37. 2025Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challengescited by 72
  38. 2024RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)cited by 68

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 22; 2026: 119.

141 papers.

Sort:

By area:

  1. 2026Causal World Modeling for Robot Controlcited by 289
  2. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  3. 2026VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Modelcited by 104
  4. 2025WMPO: World Model-based Policy Optimization for Vision-Language-Action Modelscited by 68
  5. 2026WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RLcited by 46
  6. 2026RL Token: Bootstrapping Online RL with Vision-Language-Action Modelscited by 43
  7. 2026RISE: Self-Improving Robot Policy with Compositional World Modelcited by 39
  8. 2026VLAW: Iterative Co-Improvement of Vision-Language-Action Policy and World Modelcited by 39
  9. 2025Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulationcited by 37
  10. 2026World-VLA-Loop: Closed-Loop Learning of Video World Model and VLA Policycited by 36
  11. 2026Do World Action Models Generalize Better than VLAs? A Robustness Studycited by 35
  12. 2026RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policiescited by 35
38 more
  1. 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
  2. 2026When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAscited by 27
  3. 2025RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Modelscited by 26
  4. 2025SRPO: Self-Referential Policy Optimization for Vision-Language-Action Modelscited by 25
  5. 2026World-Gymnast: Training Robots with Reinforcement Learning in a World Modelcited by 25
  6. 2025STARE-VLA: Progressive Stage-Aware Reinforcement for Fine-Tuning Vision-Language-Action Modelscited by 24
  7. 2025VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environmentscited by 22
  8. 2026AtomVLA: Scalable Post-Training for Robotic Manipulation via Predictive Latent World Modelscited by 21
  9. 2025Mixture of Horizons in Action Chunkingcited by 20
  10. 2025EVOLVE-VLA: Test-Time Training from Environment Feedback for Vision-Language-Action Modelscited by 20
  11. 2026SOP: A Scalable Online Post-Training System for Vision-Language-Action Modelscited by 19
  12. 2025DeepThinkVLA: Enhancing Reasoning Capability of Vision-Language-Action Modelscited by 17
  13. 2025π_textttRL: Online RL Fine-tuning for Flow-based Vision-Language-Action Modelscited by 16
  14. 2025AsyncVLA: Asynchronous Flow Matching for Vision-Language-Action Modelscited by 16
  15. 2026Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Modelscited by 14
  16. 2025Affordance Field Intervention: Enabling VLAs to Escape Memory Traps in Robotic Manipulationcited by 12
  17. 2026VLANeXt: Recipes for Building Strong VLA Modelscited by 12
  18. 2026Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policiescited by 12
  19. 2025EvoVLA: Self-Evolving Vision-Language-Action Modelcited by 11
  20. 2026chi0: Resource-Aware Robust Manipulation via Taming Distributional Inconsistenciescited by 11
  21. 2026Towards Long-Lived Robots: Continual Learning VLA Models via Reinforcement Fine-Tuningcited by 11
  22. 2026VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memorycited by 10
  23. 2025DEAS: DEtached value learning with Action Sequence for Scalable Offline RLcited by 9
  24. 2026VLA-OPD: Bridging Offline SFT and Online RL for Vision-Language-Action Models via On-Policy Distillationcited by 9
  25. 2026TIC-VLA: A Think-in-Control Vision-Language-Action Model for Robot Navigation in Dynamic Environmentscited by 8
  26. 2026Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Modelscited by 8
  27. 2026LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Modelscited by 8
  28. 2025VLA-4D: Embedding 4D Awareness into Vision-Language-Action Models for SpatioTemporally Coherent Robotic Manipulationcited by 7
  29. 2025Mind to Hand: Purposeful Robotic Control via Embodied Reasoningcited by 7
  30. 2026RL-VLA3: A Flexible and Asynchronous Reinforcement Learning Framework for VLA Trainingcited by 7
  31. 2026ALOE: Action-Level Off-Policy Evaluation for Vision-Language-Action Model Post-Trainingcited by 7
  32. 2026LaST-R1: Reinforcing Robotic Manipulation via Adaptive Physical Latent Reasoningcited by 7
  33. 2026EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action Modelscited by 7
  34. 2026RoboRouter: Training-Free Policy Routing for Robotic Manipulationcited by 6
  35. 2026Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Trainingcited by 6
  36. 2026NoiseGate: Learning Per-Latent Timestep Schedules as Information Gating in World Action Modelscited by 6
  37. 2026Orca: The World is in Your Mindcited by 6
  38. 2026CRL-VLA: Continual Vision-Language-Action Learningcited by 5

50 most cited of 141 All 141 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Deploying VLA without Fine-Tuning: Plug-and-Play Inference-Time VLA Policy Steering via Embodied Evolutionary Diffusion
  2. 2026SimVLA: A Simple VLA Baseline for Robotic Manipulation
  3. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey
  4. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation
  5. 2025VLA-R1: Enhancing Reasoning in Vision-Language-Action Models
  6. 2025Long-VLA: Unleashing Long-Horizon Capability of Vision Language Action Model for Robot Manipulation
  7. 2025Dual-Actor Fine-Tuning of VLA Models: A Talk-and-Tweak Human-in-the-Loop Approach
  8. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Survey