mlsanityData to 7 Oct 2026

π-StepNFT: Wider Space Needs Finer Steps in Online RL for Flow-based VLAs

arXiv 2603.02083 ↗First posted 2 Mar 2026cs.RO cs.CVcited by 5

On the map: Vision-language-action models → Efficient VLA inference and adaptation

Abstract (arXiv)

Flow-based vision-language-action (VLA) models excel in embodied control but suffer from intractable likelihoods during multi-step sampling, hindering online reinforcement learning. We propose textbftextitboldsymbolπ-StepNFT (Step-wise Negative-aware Fine-Tuning), a critic-and-likelihood-free framework that requires only a single forward pass per optimization step and eliminates auxiliary value networks. We identify that wider exploration spaces necessitate finer-grained, step-wise guidance for alignment. Empirically, π-StepNFT unlocks latent potential on LIBERO with competitive few-shot robustness. Moreover, it achieves superior generalization on ManiSkill, outperforming value-based baselines in OOD scenarios by preventing overfitting to multimodal features. This property offers a scalable solution promising for complex real-world applications.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Efficient VLA inference and adaptation, one of 13 directions in Vision-language-action models. Builds on 45 of its 51 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

45 of 51 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach2 Dec

Other references · 44

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2022Flow Matching for Generative Modelingcited by 5,426
  6. 2017Deep reinforcement learning from human preferencescited by 3,846
  7. 2015Continuous control with deep reinforcement learningcited by 3,279
  8. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  9. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  10. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  11. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  12. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
32 more
  1. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  2. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  3. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
  4. 2024PaliGemma: A versatile 3B VLM for transfercited by 679
  5. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
  6. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 507
  7. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  8. 2024PaliGemma 2: A Family of Versatile VLMs for Transfercited by 204
  9. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
  10. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
  11. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  12. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
  13. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learningcited by 141
  14. 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 102
  15. 2025Flow Matching Policy Gradientscited by 86
  16. 2025ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learningcited by 83
  17. 2025Hume: Introducing System-2 Thinking in Visual-Language-Action Modelcited by 56
  18. 2025Coefficients-Preserving Sampling for Reinforcement Learning with Flow Matchingcited by 55
  19. 2025RLinf: Flexible and Efficient Large-scale Reinforcement Learning via Macro-to-Micro Flow Transformationcited by 51
  20. 2025GigaBrain-0: A World Model-Powered Vision-Language-Action Modelcited by 43
  21. 2025GR-RL: Going Dexterous and Precise for Long-Horizon Robotic Manipulationcited by 41
  22. 2025Online Reward-Weighted Fine-Tuning of Flow Matching with Wasserstein Regularizationcited by 40
  23. 2025NFT: Bridging Supervised Learning and Reinforcement Learning in Math Reasoningcited by 38
  24. 2025Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimizationcited by 35
  25. 2025Much Ado About Noising: Dispelling the Myths of Generative Robotic Controlcited by 35
  26. 2025Reinforcement Learning for Flow-Matching Policiescited by 14
  27. 2025GCC: A 3DGS Inference Architecture with Gaussian-Wise and Cross-Stage Conditional Processingcited by 10
  28. 2025Discrete Diffusion Trajectory Alignment via Stepwise Decompositioncited by 9
  29. 2025Towards Better Optimization For Listwise Preference in Diffusion Modelscited by 7
  30. 2025Ranking-based Preference Optimization for Diffusion Models from Implicit User Feedbackcited by 3
  31. 2025Direct Diffusion Score Preference Optimization via Stepwise Contrastive Policy-Pair Supervisioncited by 2
  32. 2025Iterative Refinement of Flow Policies in Probability Space for Online Reinforcement Learningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026WCM: A World Critic Model for Vision-Language-Action Reinforcement Learningcited by 4
  2. 2026RedFlow: Redirect Failure into Action-Level Corrections for Flow-matching VLA Policycited by 3
  3. 2026StructRL: Structured Action-Space Exploration for Flow-Based VLAscited by 1
  4. 2026Where Success Breaks: Failure-Boundary Learning for Robust Vision-Language-Action Modelscited by 0
  5. 2026Uncertainty-Gated Exploration Noise Suppresses Task Collapse in Online RL Fine-Tuning of a Flow-Matching Vision-Language-Action Policycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025π_textttRL: Online RL Fine-tuning for Flow-based Vision-Language-Action Models
  2. 2025Reinforcement Fine-Tuning of Flow-Matching Policies for Vision-Language-Action Models
  3. 2026Preserving Foundational Capabilities in Flow-Matching VLAs through Conservative SFT
  4. 2026FlowPRO: Reward-Free Reinforced Fine-Tuning of Flow-Matching VLAs via Proximalized Preference Optimization
  5. 2026VLA-OPD: Bridging Offline SFT and Online RL for Vision-Language-Action Models via On-Policy Distillation
  6. 2026VLA-Precision: Asymmetric Co-Bootstrapping for Efficient Real-World Online RL of Vision-Language-Action Models
  7. 2026Q-VGM: Q-Guided Value-Gradient Matching for Offline-to-Online RL of Flow-Matching VLA Policies
  8. 2025RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Models