# Many Ways to Succeed: Diversity-Driven RL Fine-Tuning for VLA Generalization

arXiv: https://arxiv.org/abs/2610.09943 · HTML: https://mlsanity.com/paper/2610.09943/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models.
- Builds on 17 of its 41 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Placed in

- Vision-language-action models / [Reinforcement learning and post-training](https://mlsanity.com/vla/reinforcement-learning-and-post-training/)

## Builds on (most cited first, 17 of 17)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model](https://mlsanity.com/paper/2501.15830/) (2025)
- [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://mlsanity.com/paper/2506.18088/) (2025)
- [Robotic Control via Embodied Chain-of-Thought Reasoning](https://mlsanity.com/paper/2407.08693/) (2024)
- [π*0.6: a VLA That Learns From Experience](https://mlsanity.com/paper/2511.14759/) (2025)
- [LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models](https://mlsanity.com/paper/2510.13626/) (2025)
- [VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning](https://mlsanity.com/paper/2505.18719/) (2025)
- [Interactive Post-Training for Vision-Language-Action Models](https://mlsanity.com/paper/2505.17016/) (2025)
- [Do World Action Models Generalize Better than VLAs? A Robustness Study](https://mlsanity.com/paper/2603.22078/) (2026)
- [π_textttRL: Online RL Fine-tuning for Flow-based Vision-Language-Action Models](https://mlsanity.com/paper/2510.25889/) (2025)
- [Devil is in Narrow Policy: Unleashing Exploration in Driving VLA Models](https://mlsanity.com/paper/2603.06049/) (2026)
- [RESample: A Robust Data Augmentation Framework via Exploratory Sampling for Robotic Manipulation](https://mlsanity.com/paper/2510.17640/) (2025)
- [Bridging Interleaved Multi-Modal Reasoning as a Unified Decision Process](https://mlsanity.com/paper/2607.03748/) (2026)
- [Discover, Learn, and Reinforce: Scaling Vision-Language-Action Pretraining with Diverse RL-Generated Trajectories](https://mlsanity.com/paper/2511.19528/) (2025)
- [Precise: SDE-Consistent Stochastic Sampling for RL Post-Training of Flow-Matching Models](https://mlsanity.com/paper/2605.23522/) (2026)
- [Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models](https://mlsanity.com/paper/2607.03751/) (2026)
- [ExToken: Structured Exploration for Efficient Vision-Language-Action Reinforcement Fine-tuning](https://mlsanity.com/paper/2607.12931/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) fine-tuning improves vision-language-action (VLA) policies through closed-loop experience, yet generalization beyond the fine-tuning distribution remains limited. Our analysis reveals a selective reshaping of exploration: RL contracts behavior globally, yet diversifies successful trajectories, elicits success with fewer rollouts, and covers more of the latent task-valid solution space than supervised fine-tuning. Broader successful-mode coverage may provide alternative strategies under distribution shifts. Inspired by this, we introduce DRIVE (Diversity-driven RL fIne-tuning for VLA gEneralization), which turns successful-behavior diversity into an explicit RL objective. DRIVE groups rollouts under matched task conditions, compares their trajectories with temporal alignment, and derives a success-conditioned intrinsic reward from relative behavioral diversity. This design encourages broader coverage of feasible solutions without rewarding diverse failures or superficial timing differences. Across LIBERO-Plus, ManiSkill3, and RoboTwin 2.0, DRIVE improves the average out-of-domain (OOD) performance over vanilla RL fine-tuning by 5.3 points on π0 and 2.0 points on π0.5. On a dual-arm AgileX PiPER-X platform, DRIVE further increases average OOD success from 64.1% to 73.3% (+9.2 points), demonstrating gains that persist under physical deployment.