mlsanityData to 14 Oct 2026

STARE-VLA: Progressive Stage-Aware Reinforcement for Fine-Tuning Vision-Language-Action Models

arXiv 2512.05107 ↗First posted 4 Dec 2025cs.ROcited by 24

On the map: Vision-language-action models → Reinforcement learning and post-training

Abstract (arXiv)

Recent advances in Vision-Language-Action (VLA) models, powered by large language models and reinforcement learning-based fine-tuning, have shown remarkable progress in robotic manipulation. Existing methods often treat long-horizon actions as linguistic sequences and apply trajectory-level optimization methods such as Trajectory-wise Preference Optimization (TPO) or Proximal Policy Optimization (PPO), leading to coarse credit assignment and unstable training. However, unlike language, where a unified semantic meaning is preserved despite flexible sentence order, action trajectories progress through causally chained stages with different learning difficulties. This motivates progressive stage optimization. Thereby, we present Stage-Aware Reinforcement (STARE), a module that decomposes a long-horizon action trajectory into semantically meaningful stages and provides dense, interpretable, and stage-aligned reinforcement signals. Integrating STARE into TPO and PPO, we yield Stage-Aware TPO (STA-TPO) and Stage-Aware PPO (STA-PPO) for offline stage-wise preference and online intra-stage interaction, respectively. Further building on supervised fine-tuning as initialization, we propose the Imitation -> Preference -> Interaction (IPI), a serial fine-tuning pipeline for improving action accuracy in VLA models. Experiments on SimplerEnv and ManiSkill3 demonstrate substantial gains, achieving state-of-the-art success rates of 98.0 percent on SimplerEnv and 96.4 percent on ManiSkill3 tasks.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. Builds on 44 of its 48 references that are in the map. First posted on arXiv 4 Dec 2025.

Builds on

44 of 48 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025GR-RL: Going Dexterous and Precise for Long-Horizon Robotic Manipulation1 Dec
  2. 2025π*0.6: a VLA That Learns From Experience18 Nov
  3. 2025RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Models8 Oct
  4. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning11 Sep
  5. 2025What Can RL Bring to VLA Generalization? An Empirical Study26 May
  6. 2025RFTF: Reinforcement Fine-tuning for Embodied Agents with Temporal Feedback26 May
  7. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning24 May
  8. 2025Interactive Post-Training for Vision-Language-Action Models22 May
  9. 2025ReinboT: Amplifying Robot Visual-Language Manipulation with Reinforcement Learning12 May
  10. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy8 Feb
  11. 2025Improving Vision-Language-Action Model with Online Reinforcement Learning28 Jan
  12. 2024GRAPE: Generalizing Robot Policy via Preference Alignment28 Nov
1 more
  1. 2024Octo: An Open-Source Generalist Robot Policy20 May

Other references · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2023Let's Verify Step by Stepcited by 3,789
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  8. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  9. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  11. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  12. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
19 more
  1. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  2. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  3. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
  4. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  5. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 530
  6. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 519
  7. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
  8. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
  9. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  10. 2024What Matters in Building Vision-Language-Action Models for Generalist Robotscited by 89
  11. 2024Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Taskscited by 52
  12. 2025SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulationcited by 46
  13. 2025SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulationcited by 42
  14. 2025RoboFAC: A Comprehensive Framework for Robotic Failure Analysis and Correctioncited by 41
  15. 2025RoboHorizon: An LLM-Assisted Multi-View World Model for Long-Horizon Robotic Manipulationcited by 19
  16. 2025π_textttRL: Online RL Fine-tuning for Flow-based Vision-Language-Action Modelscited by 16
  17. 2024Stage-Wise Reward Shaping for Acrobatic Robots: A Constrained Multi-Objective Reinforcement Learning Approachcited by 13
  18. 2025Actor-Critic for Continuous Action Chunks: A Reinforcement Learning Framework for Long-Horizon Robotic Manipulation with Sparse Rewardcited by 13
  19. 2025Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learningcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 20.

24 papers.

Sort:

By area:

  1. 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 142
  2. 2026QVLA: Not All Channels Are Equal in Vision-Language-Action Model's Quantizationcited by 28
  3. 2026PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulationcited by 21
  4. 2025Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbonecited by 18
  5. 2026VLA-OPD: Bridging Offline SFT and Online RL for Vision-Language-Action Models via On-Policy Distillationcited by 13
  6. 2026Can Textual Reasoning Improve the Performance of MLLMs on Fine-grained Visual Classification?cited by 8
  7. 2026DTP: A Simple yet Effective Distracting Token Pruning Framework for Vision-Language Action Modelscited by 8
  8. 2026Shallow-π: Knowledge Distillation for Flow-based VLAscited by 8
  9. 2025ViBES: A Conversational Agent with Behaviorally-Intelligent 3D Virtual Bodycited by 6
  10. 2026IG-RFT: An Interaction-Guided RL Framework for VLA Models in Long-Horizon Robotic Manipulationcited by 5
  11. 2026Safe-Night VLA: Seeing the Unseen via Thermal-Perceptive Vision-Language-Action Models for Safety-Critical Manipulationcited by 5
  12. 2025CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal Reasoningcited by 4
12 more
  1. 2026Recursive Belief Vision Language Action Modelscited by 4
  2. 2026FusionAgent: A Multimodal Agent with Dynamic Model Selection for Human Recognitioncited by 4
  3. 2026Prism-GRPO: Faster VLA Policy Optimization via Splitting Same-outcome Groupscited by 2
  4. 2026Move-Then-Operate: Behavioral Phasing for Human-Like Robotic Manipulationcited by 1
  5. 2026An Open-Source Benchmark and Baseline for Multi-temporal Referring Segmentationcited by 1
  6. 2026Reward-Zero: Language Embedding Driven Implicit Reward Mechanisms for Reinforcement Learningcited by 0
  7. 2026Cross-modal Fuzzy Alignment Network for Text-Aerial Person Retrieval and A Large-scale Benchmarkcited by 0
  8. 2026PrimitiveVLA: Learning Reusable Motion Primitives for Efficient and Generalizable Robotic Manipulationcited by 0
  9. 2026Grasp-Then-Plan with Failure Attribution: A Closed Two-Stage Framework for Precise and Generalizable Robotic Manipulationcited by 0
  10. 2026Where Success Breaks: Failure-Boundary Learning for Robust Vision-Language-Action Modelscited by 0
  11. 2026RouteRLT: Learning When and Which RL Specialist Should Control a Vision-Language-Action Policycited by 0
  12. 2026Demonstration-Free Success-Probability Reward Learning for Generalist Robot Policiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026PAPO-VLA: Planning-Aware Policy Optimization for Vision-Language-Action Models
  2. 2025SRPO: Self-Referential Policy Optimization for Vision-Language-Action Models
  3. 2026SARM2: Multi-Task Stage Aware Reward Modeling for Self Improving Robotic Manipulation
  4. 2026Vision-Language-Action Jump-Starting for Reinforcement Learning Robotic Agents
  5. 2025TGRPO :Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimization
  6. 2026Z-1: Efficient Reinforcement Learning for Vision-Language-Action Models
  7. 2026SmoothVLA: Aligning Vision-Language-Action Models with Physical Constraints via Intrinsic Smoothness Optimization
  8. 2025Long-VLA: Unleashing Long-Horizon Capability of Vision Language Action Model for Robot Manipulation