mlsanityData to 7 Oct 2026

Interactive Post-Training for Vision-Language-Action Models

arXiv 2505.17016 ↗First posted 22 May 2025cs.LG cs.AI cs.CVcited by 129

On the map: Vision-language-action models → Reinforcement learning and post-training

Abstract (arXiv)

We introduce RIPT-VLA, a simple and scalable reinforcement-learning-based interactive post-training paradigm that fine-tunes pretrained Vision-Language-Action (VLA) models using only sparse binary success rewards. Existing VLA training pipelines rely heavily on offline expert demonstration data and supervised imitation, limiting their ability to adapt to new tasks and environments under low-data regimes. RIPT-VLA addresses this by enabling interactive post-training with a stable policy optimization algorithm based on dynamic rollout sampling and leave-one-out advantage estimation. RIPT-VLA has the following characteristics. First, it applies to various VLA models, resulting in an improvement on the lightweight QueST model by 21.2%, and the 7B OpenVLA-OFT model to an unprecedented 97.5% success rate. Second, it is computationally efficient and data-efficient: with only one demonstration, RIPT-VLA enables an unworkable SFT model (4%) to succeed with a 97% success rate within 15 iterations. Furthermore, we demonstrate that the policy learned by RIPT-VLA generalizes across different tasks and scenarios and is robust to the initial state context. These results highlight RIPT-VLA as a practical and effective paradigm for post-training VLA models through minimal supervision.

Key facts. Cited by 129 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. Builds on 34 of its 41 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

34 of 41 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy8 Feb
  2. 2025Improving Vision-Language-Action Model with Online Reinforcement Learning28 Jan
  3. 2024Octo: An Open-Source Generalist Robot Policy20 May

Other references · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  8. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  9. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  10. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  11. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  12. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
19 more
  1. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  2. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  3. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  4. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  6. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  7. 2018Attention, Learn to Solve Routing Problems!cited by 646
  8. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  9. 2024Behavior Generation with Latent Actionscited by 159
  10. 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulationcited by 159
  11. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  12. 2024Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goalscited by 131
  13. 2024Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planningcited by 95
  14. 2025Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policycited by 73
  15. 2024QueST: Self-Supervised Skill Abstractions for Learning Continuous Controlcited by 51
  16. 2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 40
  17. 2023ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioningcited by 24
  18. 2024PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Controlcited by 21
  19. 2024Direct Large Language Model Alignment Through Self-Rewarding Contrastive Prompt Distillationcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 31; 2026: 98.

129 papers.

Sort:

By area:

  1. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  2. 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Modelscited by 278
  3. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  4. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learningcited by 141
  5. 2026ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learningcited by 94
  6. 2025Self-Improving Vision-Language-Action Models with Data Generation via Residual RLcited by 72
  7. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  8. 2025VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulatorscited by 65
  9. 2025World-Env: Leveraging World Model as a Virtual Environment for VLA Post-Trainingcited by 57
  10. 2025Residual Off-Policy RL for Finetuning Behavior Cloning Policiescited by 42
  11. 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
  12. 2026ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Modelscited by 30
38 more
  1. 2025VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Modelscited by 28
  2. 2026ABot-M0.5: Unified Mobility-and-Manipulation World Action Modelcited by 28
  3. 2026StarVLA-α: Reducing Complexity in Vision-Language-Action Systemscited by 27
  4. 2025RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Modelscited by 26
  5. 2025SRPO: Self-Referential Policy Optimization for Vision-Language-Action Modelscited by 25
  6. 2025STARE-VLA: Progressive Stage-Aware Reinforcement for Fine-Tuning Vision-Language-Action Modelscited by 24
  7. 2026GigaBrain-0.5M*: a VLA That Learns From World Model-Based Reinforcement Learningcited by 23
  8. 2025AVA-VLA: Improving Vision-Language-Action models with Active Visual Attentioncited by 22
  9. 2025RLRC: Reinforcement Learning-based Recovery for Compressed Vision-Language-Action Modelscited by 20
  10. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  11. 2025DeepThinkVLA: Enhancing Reasoning Capability of Vision-Language-Action Modelscited by 17
  12. 2025π_textttRL: Online RL Fine-tuning for Flow-based Vision-Language-Action Modelscited by 16
  13. 2025RobustVLA: Robustness-Aware Reinforcement Post-Training for Vision-Language-Action Modelscited by 16
  14. 2025Balancing Signal and Variance: Adaptive Offline RL Post-Training for VLA Flow Modelscited by 14
  15. 2026On-the-Fly VLA Adaptation via Test-Time Reinforcement Learningcited by 14
  16. 2026Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Modelscited by 14
  17. 2026VLANeXt: Recipes for Building Strong VLA Modelscited by 12
  18. 2026Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policiescited by 12
  19. 2026Towards Long-Lived Robots: Continual Learning VLA Models via Reinforcement Fine-Tuningcited by 11
  20. 2026Geometric Action Model for Robot Policy Learningcited by 10
  21. 2026ST-WAM: Semantic-Temporal World Action Model for Robust Manipulation under Visual Distribution Shiftscited by 10
  22. 2025DEAS: DEtached value learning with Action Sequence for Scalable Offline RLcited by 9
  23. 2025When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Modelscited by 9
  24. 2026VLA-OPD: Bridging Offline SFT and Online RL for Vision-Language-Action Models via On-Policy Distillationcited by 9
  25. 2026MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Modelscited by 9
  26. 2026Flex-π: A Multi-Stream World-Action Model with Compute Flexibilitycited by 9
  27. 2025Reinforcement Learning for Large Model: A Surveycited by 8
  28. 2026SafeVLA-Bench: A Benchmark for the Success-Safety Gap in Vision-Language-Action Modelscited by 8
  29. 2026LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Modelscited by 8
  30. 2025Survey of Vision-Language-Action Models for Embodied Manipulationcited by 7
  31. 2026RL-VLA3: A Flexible and Asynchronous Reinforcement Learning Framework for VLA Trainingcited by 7
  32. 2026ALOE: Action-Level Off-Policy Evaluation for Vision-Language-Action Model Post-Trainingcited by 7
  33. 2026World2Act: Latent Action Post-Training from World Model Dynamicscited by 7
  34. 2026LaST-R1: Reinforcing Robotic Manipulation via Adaptive Physical Latent Reasoningcited by 7
  35. 2026GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specializationcited by 7
  36. 2026EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action Modelscited by 7
  37. 2025Asynchronous Fast-Slow Vision-Language-Action Policies for Whole-Body Robotic Manipulationcited by 6
  38. 2026OmniGuide: Universal Guidance Fields for Enhancing Generalist Robot Policiescited by 6

50 most cited of 129 All 129 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025VLA Model Post-Training via Action-Chunked PPO and Self Behavior Cloning
  2. 2025VLA-R1: Enhancing Reasoning in Vision-Language-Action Models
  3. 2026Stable and Efficient Real-World Online VLA Post-Training via Asynchronous Replay-Anchored Policy Improvement
  4. 2026SOP: A Scalable Online Post-Training System for Vision-Language-Action Models
  5. 2026Agentic-VLA: Efficient Online Adaptation for Vision-Language-Action Models
  6. 2026TTT-VLA: Test-Time Latent Prompt Optimization for Vision-Language-Action Models
  7. 2026Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models
  8. 2025Enhancing Vision-Language Model Training with Reinforcement Learning in Synthetic Worlds for Real-World Success