mlsanityData to 7 Oct 2026

Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learning

arXiv 2603.11653 ↗First posted 12 Mar 2026cs.LG cs.ROcited by 13

On the map: Vision-language-action models → Reinforcement learning and post-training

Abstract (arXiv)

Continual Reinforcement Learning (CRL) for Vision-Language-Action (VLA) models is a promising direction toward self-improving embodied agents that can adapt in openended, evolving environments. However, conventional wisdom from continual learning suggests that naive Sequential Fine-Tuning (Seq. FT) leads to catastrophic forgetting, necessitating complex CRL strategies. In this work, we take a step back and conduct a systematic study of CRL for large pretrained VLAs across diverse lifelong RL benchmarks. We find that, contrary to established belief, simple Seq. FT with low-rank adaptation (LoRA) is remarkably strong: it achieves high plasticity, exhibits little to no forgetting, and retains strong zero-shot generalization, frequently outperforming more sophisticated CRL methods. Through detailed analysis, we show that this robustness arises from a synergy between the large pretrained model, parameter-efficient adaptation, and on-policy RL. Together, these components reshape the stability-plasticity trade-off, making continual adaptation both stable and scalable. Our results position Sequential Fine-Tuning as a powerful method for continual RL with VLAs and provide new insights into lifelong learning in the large model era. Code is available at https://github.com/UT-Austin-RobIn/continual-vla-rl

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. Builds on 53 of its 64 references that are in the map. First posted on arXiv 12 Mar 2026.

Builds on

53 of 64 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning4 Mar
  2. 2025Robust Finetuning of Vision-Language-Action Robot Policies via Parameter Merging9 Dec
  3. 2025π*0.6: a VLA That Learns From Experience18 Nov
  4. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning24 May

Other references · 49

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  8. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  9. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  10. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  11. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  12. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
37 more
  1. 2016Progressive Neural Networkscited by 1,332
  2. 2018Experience Replay for Continual Learningcited by 1,074
  3. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
  4. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  5. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  6. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  7. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  8. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  9. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 391
  10. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
  11. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  12. 2024LoRA Learns Less and Forgets Lesscited by 313
  13. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
  14. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
  15. 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 178
  16. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
  17. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
  18. 2022On the Effectiveness of Parameter-Efficient Fine-Tuningcited by 140
  19. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 118
  20. 2023Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Modelscited by 112
  21. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 111
  22. 2023A Definition of Continual Reinforcement Learningcited by 98
  23. 2025Retaining by Doing: The Role of On-Policy Data in Mitigating Forgettingcited by 82
  24. 2021Wide Neural Networks Forget Less Catastrophicallycited by 64
  25. 2024FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuningcited by 54
  26. 2025RLinf: Flexible and Efficient Large-scale Reinforcement Learning via Macro-to-Micro Flow Transformationcited by 51
  27. 2023TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Modelscited by 48
  28. 2025Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Trainingcited by 42
  29. 2019Policy Consolidation for Continual Reinforcement Learningcited by 37
  30. 2021Lifelong Robotic Reinforcement Learning by Retaining Experiencescited by 34
  31. 2022Disentangling Transfer in Continual Reinforcement Learningcited by 31
  32. 2025Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churncited by 22
  33. 2025A Survey of Continual Reinforcement Learningcited by 16
  34. 2021CoMPS: Continual Meta Policy Searchcited by 15
  35. 2022The Effectiveness of World Models for Continual Reinforcement Learningcited by 15
  36. 2025Rethinking the Foundations for Continual Reinforcement Learningcited by 15
  37. 2024Position: Lifetime tuning is incompatible with continual reinforcement learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area:

  1. 2026What Matters in Orchestrating Robot Policies: A Systematic Study of Hierarchical VLA Agentscited by 9
  2. 2026Can Vision-Language-Action Models Learn from Real-World Data Continually without Forgetting?cited by 4
  3. 2026DiscreteRTC: Discrete Diffusion Policies are Natural Asynchronous Executorscited by 3
  4. 2026Preserving Foundational Capabilities in Flow-Matching VLAs through Conservative SFTcited by 1
  5. 2026PHASER: Phase-Aware and Semantic Experience Replay for Vision-Language-Action Modelscited by 1
  6. 2026Beyond One-Size-Fits-All: Diagnosis-Driven Online Reinforcement Learning with Offline Priorscited by 0
  7. 2026Taming the Implicit: Dual-Channel Risk-Aware Reinforcement Fine-Tuning for Continual Multimodal Post-Trainingcited by 0
  8. 2026PhaseLoRA: Control-Regime-Conditioned Low-Rank Adaptation for Continuous-Action Vision-Language-Action Policiescited by 0
  9. 2026Fine-Tuning VLAs with Self-Demonstrated Generative Control for Multi-Task Manipulationcited by 0
  10. 2026FAN: Foresight Action Normalization for Continual Adaptation of Vision-Language-Action Modelscited by 0
  11. 2026Uncertainty-Gated Exploration Noise Suppresses Task Collapse in Online RL Fine-Tuning of a Flow-Matching Vision-Language-Action Policycited by 0
  12. 2026SAMBAR: Selective Anchoring via Method of Multipliers for Balanced Knowledge Acquisition and Retention in Vision-Language-Action Modelscited by 0
1 more
  1. 2026Train Together or Merge Later? Unifying VLA Experts via a Shared Action Interfacecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CRL-VLA: Continual Vision-Language-Action Learning
  2. 2025Continual Learning in Vision-Language Models via Aligned Model Merging
  3. 2026RL Forgets! Towards Continual Policy Optimization
  4. 2026Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuning
  5. 2026Towards Long-Lived Robots: Continual Learning VLA Models via Reinforcement Fine-Tuning
  6. 2023Continual Visual Reinforcement Learning with A Life-Long World Model
  7. 2026RECALL: Recovery Experience Collection for Active Lifelong Learning in Vision-Language-Action Models
  8. 2025Synthetic Data is an Elegant GIFT for Continual Vision-Language Models