# Towards Long-Lived Robots: Continual Learning VLA Models via Reinforcement Fine-Tuning arXiv: https://arxiv.org/abs/2602.10503 · HTML: https://mlsanity.com/paper/2602.10503/ ## Key facts - Cited by 13 papers in the map (data through 14 Oct 2026). - Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. - Builds on 73 of its 81 references that are in the map. - First posted on arXiv 11 Feb 2026. ## Placed in - Vision-language-action models / [Reinforcement learning and post-training](https://mlsanity.com/vla/reinforcement-learning-and-post-training/) ## Builds on (most cited first, 30 of 73) - [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017) - [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024) - [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025) - [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017) - [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023) - [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024) - [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024) - [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022) - [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025) - [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024) - [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023) - [GR00T N1: An Open Foundation Model for Generalist Humanoid Robots](https://mlsanity.com/paper/2503.14734/) (2025) - [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017) - [Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success](https://mlsanity.com/paper/2502.19645/) (2025) - [PaliGemma: A versatile 3B VLM for transfer](https://mlsanity.com/paper/2407.07726/) (2024) - [FAST: Efficient Action Tokenization for Vision-Language-Action Models](https://mlsanity.com/paper/2501.09747/) (2025) - [BridgeData V2: A Dataset for Robot Learning at Scale](https://mlsanity.com/paper/2308.12952/) (2023) - [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2308.08747/) (2023) - [SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model](https://mlsanity.com/paper/2501.15830/) (2025) - [Evaluating Real-World Robot Manipulation Policies in Simulation](https://mlsanity.com/paper/2405.05941/) (2024) - [CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models](https://mlsanity.com/paper/2503.22020/) (2025) - [WorldVLA: Towards Autoregressive Action World Model](https://mlsanity.com/paper/2506.21539/) (2025) - [TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policies](https://mlsanity.com/paper/2412.10345/) (2024) - [X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model](https://mlsanity.com/paper/2510.10274/) (2025) - [Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1904.00310/) (2019) - [Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2004.10956/) (2020) - [MolmoAct: Action Reasoning Models that can Reason in Space](https://mlsanity.com/paper/2508.07917/) (2025) - [Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning](https://mlsanity.com/paper/2410.21845/) (2024) - [VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning](https://mlsanity.com/paper/2505.18719/) (2025) - [RL's Razor: Why Online Reinforcement Learning Forgets Less](https://mlsanity.com/paper/2509.04259/) (2025) ## Built on by (most cited first, 13 of 13) - [CORAL: Scalable Multi-Task Robot Learning via LoRA Experts](https://mlsanity.com/paper/2603.09298/) (2026) - [Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Models](https://mlsanity.com/paper/2605.13119/) (2026) - [Can Vision-Language-Action Models Learn from Real-World Data Continually without Forgetting?](https://mlsanity.com/paper/2605.26820/) (2026) - [World Action Models Enable Continual Imitation Learning with Recurrent Generative Replays](https://mlsanity.com/paper/2606.27374/) (2026) - [Preserving Foundational Capabilities in Flow-Matching VLAs through Conservative SFT](https://mlsanity.com/paper/2605.08879/) (2026) - [Self-Evolving AI for Humanoids: Mechanisms, Safety, and Evaluation of Post-Deployment Self-Improvement](https://mlsanity.com/paper/2609.13236/) (2026) - [Closing the Loop: Unified 3D Scene Generation and Immersive Interaction via LLM-RL Coupling](https://mlsanity.com/paper/2605.05711/) (2026) - [Fine-Tuning VLAs with Self-Demonstrated Generative Control for Multi-Task Manipulation](https://mlsanity.com/paper/2608.19490/) (2026) - [FAN: Foresight Action Normalization for Continual Adaptation of Vision-Language-Action Models](https://mlsanity.com/paper/2609.21358/) (2026) - [SAMBAR: Selective Anchoring via Method of Multipliers for Balanced Knowledge Acquisition and Retention in Vision-Language-Action Models](https://mlsanity.com/paper/2609.32108/) (2026) - [Train Together or Merge Later? Unifying VLA Experts via a Shared Action Interface](https://mlsanity.com/paper/2609.33125/) (2026) - [Demonstration-Free Success-Probability Reward Learning for Generalist Robot Policies](https://mlsanity.com/paper/2609.33653/) (2026) - [ACCRUE: Continual Off-Policy Reinforcement Learning for Robot Foundation Models](https://mlsanity.com/paper/2610.18596/) (2026) ## Abstract (arXiv) Pretrained on large-scale and diverse datasets, VLA models demonstrate strong generalization and adaptability as general-purpose robotic policies. However, Supervised Fine-Tuning (SFT), which serves as the primary mechanism for adapting VLAs to downstream domains, requires substantial amounts of task-specific data and is prone to catastrophic forgetting. To address these limitations, we propose LifeLong-RFT, a simple yet effective Reinforcement Fine-Tuning (RFT) strategy for VLA models independent of online environmental feedback and pre-trained reward models. By integrating chunking-level on-policy reinforcement learning with the proposed multi-dimensional process reward mechanism, LifeLong-RFT quantifies the heterogeneous contributions of intermediate action chunks across three dimensions to facilitate policy optimization. Specifically, (1) the Quantized Action Consistency Reward (QACR) ensures accurate action prediction within the discrete action space; (2) the Continuous Trajectory Alignment Reward (CTAR) aligns decoded continuous action chunks with reference trajectories to ensure precise control; (3) the Format Compliance Reward (FCR) guarantees the structural validity of outputs. Comprehensive experiments across SimplerEnv, LIBERO, and real-world tasks demonstrate that LifeLong-RFT exhibits strong performance in multi-task learning. Furthermore, for continual learning on the LIBERO benchmark, our method achieves a 22% gain in average success rate over SFT, while effectively adapting to new tasks using only 20% of the training data. Overall, our method provides a promising post-training paradigm for VLAs. The project page is available at .