mlsanityData to 7 Oct 2026

π*0.6: a VLA That Learns From Experience

arXiv 2511.14759 ↗First posted 18 Nov 2025cs.LG cs.ROcited by 280

On the map: Vision-language-action models → Reinforcement learning and post-training

Abstract (arXiv)

We study how vision-language-action (VLA) models can improve through real-world deployments via reinforcement learning (RL). We present a general-purpose method, RL with Experience and Corrections via Advantage-conditioned Policies (RECAP), that provides for RL training of VLAs via advantage conditioning. Our method incorporates heterogeneous data into the self-improvement process, including demonstrations, data from on-policy collection, and expert teleoperated interventions provided during autonomous execution. RECAP starts by pre-training a generalist VLA with offline RL, which we call π*0.6, that can then be specialized to attain high performance on downstream tasks through on-robot data collection. We show that the π*0.6 model trained with the full RECAP method can fold laundry in real homes, reliably assemble boxes, and make espresso drinks using a professional espresso machine. On some of the hardest tasks, RECAP more than doubles task throughput and roughly halves the task failure rate.

Key facts. Cited by 280 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. Builds on 69 of its 86 references that are in the map. First posted on arXiv 18 Nov 2025.

Builds on

69 of 86 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Self-Improving Vision-Language-Action Models with Data Generation via Residual RL30 Oct
  2. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learning19 Sep
  3. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning11 Sep
  4. 2025Align-Then-stEer: Adapting the Vision-Language Action Models through Unified Latent Guidance2 Sep
  5. 2025CO-RFT: Efficient Fine-Tuning of Vision-Language-Action Models through Chunked Offline Reinforcement Learning4 Aug
  6. 2025What Can RL Bring to VLA Generalization? An Empirical Study26 May
  7. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning24 May
  8. 2025Interactive Post-Training for Vision-Language-Action Models22 May
  9. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy8 Feb
  10. 2025Improving Vision-Language-Action Model with Online Reinforcement Learning28 Jan
  11. 2024GRAPE: Generalizing Robot Policy via Preference Alignment28 Nov
  12. 2024Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance17 Oct

Other references · 57

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2022Flow Matching for Generative Modelingcited by 5,426
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  6. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  8. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  9. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  10. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  11. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  12. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
38 more
  1. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  2. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  3. 2018Maximum a Posteriori Policy Optimisationcited by 318
  4. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  5. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 293
  6. 2020Critic Regularized Regressioncited by 269
  7. 2024Diffusion Policy Policy Optimizationcited by 255
  8. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 245
  9. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  10. 2022Online Decision Transformercited by 194
  11. 2017DART: Noise Injection for Robust Imitation Learningcited by 187
  12. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 184
  13. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 180
  14. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
  15. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 142
  16. 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policiescited by 128
  17. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 124
  18. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 117
  19. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  20. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 111
  21. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  22. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RLcited by 103
  23. 2019Reward-Conditioned Policiescited by 101
  24. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
  25. 2019IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Datacited by 85
  26. 2024Vision Language Models are In-Context Value Learnerscited by 84
  27. 2022When does return-conditioned supervised learning work for offline reinforcement learning?cited by 73
  28. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 71
  29. 2023Elastic Decision Transformercited by 66
  30. 2025ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrationscited by 52
  31. 2025Diffusion Guidance Is a Controllable Policy Improvement Operatorcited by 51
  32. 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 49
  33. 2025RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learningcited by 47
  34. 2024RLDG: Robotic Generalist Policy Distillation via Reinforcement Learningcited by 42
  35. 2025Residual Off-Policy RL for Finetuning Behavior Cloning Policiescited by 42
  36. 2025RaC: Robot Learning for Long-Horizon Tasks by Scaling Recovery and Correctioncited by 35
  37. 2021Autonomous Reinforcement Learning: Formalism and Benchmarkingcited by 33
  38. 2024Simple Policy Optimizationcited by 31

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 14; 2026: 266.

280 papers.

Sort:

By area:

  1. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
  2. 2026DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Controlcited by 71
  3. 2025Training-Time Action Conditioning for Efficient Real-Time Chunkingcited by 61
  4. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 51
  5. 2026RL Token: Bootstrapping Online RL with Vision-Language-Action Modelscited by 43
  6. 2026World Model for Robot Learning: A Comprehensive Surveycited by 42
  7. 2025GR-RL: Going Dexterous and Precise for Long-Horizon Robotic Manipulationcited by 41
  8. 2026RISE: Self-Improving Robot Policy with Compositional World Modelcited by 39
  9. 2026VLAW: Iterative Co-Improvement of Vision-Language-Action Policy and World Modelcited by 39
  10. 2026World-VLA-Loop: Closed-Loop Learning of Video World Model and VLA Policycited by 36
  11. 2026World Guidance: World Modeling in Condition Space for Action Generationcited by 32
  12. 2026ENPIRE: Agentic Robot Policy Self-Improvement in the Real Worldcited by 31
38 more
  1. 2026TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Roboticscited by 29
  2. 2026CLAP: Contrastive Latent Action Pretraining for Learning Vision-Language-Action Models from Human Videoscited by 27
  3. 2025MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learningcited by 25
  4. 2026RLDX-1 Technical Reportcited by 25
  5. 2025STARE-VLA: Progressive Stage-Aware Reinforcement for Fine-Tuning Vision-Language-Action Modelscited by 24
  6. 2026Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectoriescited by 24
  7. 2026GigaBrain-0.5M*: a VLA That Learns From World Model-Based Reinforcement Learningcited by 23
  8. 2025EVOLVE-VLA: Test-Time Training from Environment Feedback for Vision-Language-Action Modelscited by 20
  9. 2026PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulationcited by 20
  10. 2026HoloBrain-0 Technical Reportcited by 20
  11. 2026FASTER: Rethinking Real-Time Flow VLAscited by 20
  12. 2026Hy-Embodied-0.5-VLA: From Vision-Language-Action Models to a Real-World Robot Learning Stackcited by 20
  13. 2025Robust Finetuning of Vision-Language-Action Robot Policies via Parameter Mergingcited by 19
  14. 2026SOP: A Scalable Online Post-Training System for Vision-Language-Action Modelscited by 19
  15. 2026EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policiescited by 18
  16. 2026ActionCodec: What Makes for Good Action Tokenizerscited by 17
  17. 2026How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perfcited by 17
  18. 2026Action-to-Action Flow Matchingcited by 15
  19. 2026AtomicVLA: Unlocking the Potential of Atomic Skill Learning in Robotscited by 15
  20. 2026SpanVLA: Efficient Action Bridging and Learning from Negative-Recovery Samples for Vision-Language-Action Modelcited by 15
  21. 2026Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Modelscited by 14
  22. 2026Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learningcited by 13
  23. 2026Evo-Depth: A Lightweight Depth-Enhanced Vision-Language-Action Modelcited by 13
  24. 2026VLANeXt: Recipes for Building Strong VLA Modelscited by 12
  25. 2026LOME: Learning Human-Object Manipulation with Action-Conditioned Egocentric World Modelcited by 12
  26. 2026Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policiescited by 12
  27. 2026chi0: Resource-Aware Robust Manipulation via Taming Distributional Inconsistenciescited by 11
  28. 2026Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Modelscited by 11
  29. 2025GLaD: Geometric Latent Distillation for Vision-Language-Action Modelscited by 10
  30. 2026ARM: Advantage Reward Modeling for Long-Horizon Manipulationcited by 10
  31. 2026Action Images: End-to-End Policy Learning via Multiview Video Generationcited by 10
  32. 2026Is the Future Compatible? Diagnosing Dynamic Consistency in World Action Modelscited by 10
  33. 2026GigaWorld-Policy-0.5: A Faster and Stronger WAM Empowered by AutoResearchcited by 10
  34. 2026VLA-OPD: Bridging Offline SFT and Online RL for Vision-Language-Action Models via On-Policy Distillationcited by 9
  35. 2026Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Modelscited by 8
  36. 2026ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulationcited by 8
  37. 2026SpatialVAM:Spatial-Aware Multi-View Video Diffusion as a Data-Efficient Robot Policycited by 8
  38. 2026ProcVLM: Learning Procedure-Grounded Progress Rewards for Robotic Manipulationcited by 8

50 most cited of 280 All 280 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026On-the-Fly VLA Adaptation via Test-Time Reinforcement Learning
  2. 2025Reinforcing Action Policies by Prophesying
  3. 2026RL-VLA3: A Flexible and Asynchronous Reinforcement Learning Framework for VLA Training
  4. 2026Vision-Language-Action Jump-Starting for Reinforcement Learning Robotic Agents
  5. 2026Agentic-VLA: Efficient Online Adaptation for Vision-Language-Action Models
  6. 2026DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies
  7. 2025RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Models
  8. 2025VLA-R1: Enhancing Reasoning in Vision-Language-Action Models