mlsanityData to 14 Oct 2026

RedFlow: Redirect Failure into Action-Level Corrections for Flow-matching VLA Policy

arXiv 2607.27782 ↗First posted 30 Jul 2026cs.RO cs.AIcited by 4

On the map: Vision-language-action models → Efficient VLA inference and adaptation

also in 2 more
Abstract (arXiv)

Reinforcement learning (RL) can improve Vision-Language-Action (VLA) policies from deployment experience, but reward- and preference-based RL primarily identifies desirable behaviors without specifying how to correct failed actions, underutilizing failure trajectories and limiting sample efficiency. Can such corrections be derived from fixed rollouts? Our key insight is that rollouts with different outcomes may contain action chunks executed in similar states, enabling higher-quality chunks to provide locally supported corrective references. Building on this insight, we introduce RedFlow, an offline post-training method for flow-matching VLA policies. Execution-Context Matching groups chunks using a compact representation of estimated task progress and robot proprioception. Quality-Guided Action Redirection assigns signed chunk-quality scores and aggregates higher-quality chunks into corrective targets, reinforcing high-quality chunks, suppressing low-quality chunks, and redirecting correctable chunks toward their targets. RedFlow requires neither external HIL corrections nor online data collection during post-training. Across four LIBERO suites, RedFlow improves average success from 56.2% to 68.2%, outperforming the strongest evaluated offline baseline, AWR (62.3%), by 5.9 points. Across three real-robot tasks, it improves average success from 56.7% to 74.7%. On LIBERO-Spatial, RedFlow reaches 75.8% success with 1,536 fixed rollouts, while the evaluated online methods require 8.7--16× as many fresh post-training rollouts to reach the same threshold.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Efficient VLA inference and adaptation, one of 13 directions in Vision-language-action models. Builds on 45 of its 53 references that are in the map. First posted on arXiv 30 Jul 2026.

Builds on

45 of 53 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Q-VGM: Q-Guided Value-Gradient Matching for Offline-to-Online RL of Flow-Matching VLA Policies6 Jun
  2. 2026FlowPRO: Reward-Free Reinforced Fine-Tuning of Flow-Matching VLAs via Proximalized Preference Optimization3 Jun
  3. 2026Potential-Guided Flow Matching for Vision-Language-Action Policy Improvement3 Jun
  4. 2026π-StepNFT: Wider Space Needs Finer Steps in Online RL for Flow-based VLAs2 Mar
  5. 2025Balancing Signal and Variance: Adaptive Offline RL Post-Training for VLA Flow Models4 Sep

Other references · 40

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2022Flow Matching for Generative Modelingcited by 5,702
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  7. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  8. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  9. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  10. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  11. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  12. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
28 more
  1. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  2. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  3. 2025π*0.6: a VLA That Learns From Experiencecited by 303
  4. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 266
  5. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learningcited by 189
  6. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 176
  7. 2024GRAPE: Generalizing Robot Policy via Preference Alignmentcited by 109
  8. 2017Accelerated Hierarchical Density Clusteringcited by 102
  9. 2025ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learningcited by 97
  10. 2025WMPO: World Model-based Policy Optimization for Vision-Language-Action Modelscited by 70
  11. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 60
  12. 2025RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learningcited by 59
  13. 2025Diffusion Guidance Is a Controllable Policy Improvement Operatorcited by 58
  14. 2025RLinf: Flexible and Efficient Large-scale Reinforcement Learning via Macro-to-Micro Flow Transformationcited by 57
  15. 2024Exploring Prosocial Irrationality for LLM Agents: A Social Cognition Viewcited by 38
  16. 2025CO-RFT: Efficient Fine-Tuning of Vision-Language-Action Models through Chunked Offline Reinforcement Learningcited by 37
  17. 2024DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitchingcited by 36
  18. 2025A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Searchcited by 17
  19. 2026HALO: A Unified Vision-Language-Action Model for Embodied Multimodal Chain-of-Thought Reasoningcited by 16
  20. 2022Model-based Trajectory Stitching for Improved Offline Reinforcement Learningcited by 15
  21. 2025Human-assisted Robotic Policy Refinement via Action Preference Optimizationcited by 13
  22. 2026chi0: Resource-Aware Robust Manipulation via Taming Distributional Inconsistenciescited by 12
  23. 2025United Minds or Isolated Agents? Exploring Coordination of LLMs under Cognitive Load Theorycited by 10
  24. 2026From Prior to Pro: Efficient Skill Mastery via Distribution Contractive RL Finetuningcited by 10
  25. 2025CoBRA: Programming Cognitive Bias in Social Agents Using Classic Social Science Experimentscited by 8
  26. 2025Love First, Know Later: Persona-Based Romantic Compatibility Through LLM Text World Enginescited by 3
  27. 2026VISTA: Enhancing Visual Conditioning via Track-Following Preference Optimization in Vision-Language-Action Modelscited by 3
  28. 2026Hand-in-the-Loop: Improving VLA Policies for Dexterous Manipulation via Seamless Hand-Arm Interventioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Rethinking Demonstration Unlearning in Imitation Learning for Roboticscited by 1
  2. 2026Causal-History Test-Time Scaling for Failure Recovery in Autoregressive World-Action Modelscited by 1
  3. 2026PAVE: Predictive Alignment and Value-Guided Evolution for World-Action Policiescited by 0
  4. 2026MedEvo: Medical Reasoning Evolution through Knowledge Navigationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Fewer Steps, Better Actions: Rethinking Flow-Matching Inference for VLA Policies
  2. 2026RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Models
  3. 2026Failing Forward: Adaptive Failure-Informed Learning for Vision-Language-Action Models
  4. 2026FailPatch: Failure Residual Patching for Vision-Language-Action Models
  5. 2026SA-VLA: Spatially-Aware Flow-Matching for Vision-Language-Action Reinforcement Learning
  6. 2026TraceFlow: Guiding Frozen Flow-Matching Robot Policies with Success and Failure Traces
  7. 2025RobustVLA: Robustness-Aware Reinforcement Post-Training for Vision-Language-Action Models
  8. 2026Learning from Runtime Feedback through Failure-Bank Self-Evolution for Vision-Language-Action Models