mlsanityData to 14 Oct 2026

DistAL: Distance-based Advantage Learning for VLA Fine-Tuning

arXiv 2609.18392 ↗First posted 16 Sep 2026cs.ROcited by 0

On the map: Vision-language-action models → Reinforcement learning and post-training

also in 1 more
Abstract (arXiv)

Vision-language-action models (VLAs) have trans- formed the field of robotic manipulation in recent years by combining the semantic understanding of LLMs with the precise control of flow-matching policies. Advantage conditioning is a recent technique that iteratively improves VLAs by training a value function on deployment data and using this to train an advantage-conditioned policy. Previous works have only applied simple, low-information success/failure rewards, which leave the value function unable to distinguish states of differing quality beyond how far along the task they appear. Motivated by an exploration of out-of-distribution (OOD) detection methods, we introduce Distance-based Advantage Learning (DistAL), which, by using an embedding space distance as a reward, produces a more informative value function and subsequently a higher downstream task success rate. We validate our method on a series of simulation benchmarks and dexterous bi-manual manipulation tasks on real hardware.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. Builds on 43 of its 46 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

43 of 46 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025π*0.6: a VLA That Learns From Experience18 Nov
  2. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning24 May
  3. 2025Interactive Post-Training for Vision-Language-Action Models22 May
  4. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy8 Feb
  5. 2025Improving Vision-Language-Action Model with Online Reinforcement Learning28 Jan
  6. 2024Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance17 Oct
  7. 2024Octo: An Open-Source Generalist Robot Policy20 May

Other references · 36

  1. 2022Classifier-Free Diffusion Guidancecited by 5,847
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  4. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  8. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  9. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  10. 2025Gemma 3 Technical Reportcited by 1,824
  11. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  12. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
24 more
  1. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  2. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  3. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  5. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  6. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 883
  7. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  8. 2024PaliGemma: A versatile 3B VLM for transfercited by 696
  9. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 691
  10. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 600
  11. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 422
  12. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 308
  13. 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Modelscited by 303
  14. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learningcited by 189
  15. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 144
  16. 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 108
  17. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 79
  18. 2024Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progresscited by 69
  19. 2024Real-Time Anomaly Detection and Reactive Planning with Large Language Modelscited by 68
  20. 2025Diffusion Guidance Is a Controllable Policy Improvement Operatorcited by 58
  21. 2025Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policiescited by 52
  22. 2024RLDG: Robotic Generalist Policy Distillation via Reinforcement Learningcited by 47
  23. 2024Multi-Task Interactive Robot Fleet Learning with Visual World Modelscited by 36
  24. 2022STAP: Sequencing Task-Agnostic Policiescited by 18

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026VISTA: Enhancing Visual Conditioning via Track-Following Preference Optimization in Vision-Language-Action Models
  2. 2025MoS-VLA: A Vision-Language-Action Model with One-Shot Skill Adaptation
  3. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control
  4. 2026Dissecting Advantage-Guided Post-Training for Vision-Language-Action Policies
  5. 2026Fine-Tuning VLAs with Self-Demonstrated Generative Control for Multi-Task Manipulation
  6. 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks
  7. 2025Long-VLA: Unleashing Long-Horizon Capability of Vision Language Action Model for Robot Manipulation
  8. 2026BORA: Bridging Offline Reinforcement Learning and Online Residual Adaptation for Real-World Dexterous VLA Models