mlsanityData to 7 Oct 2026

Improving Robotic Generalist Policies via Flow Reversal Steering

arXiv 2606.13675 ↗First posted 11 Jun 2026cs.ROcited by 5

On the map: Vision-language-action models → Reinforcement learning and post-training

Abstract (arXiv)

Generalist policies can learn a wide range of skills from diverse robot datasets. In order to solve or improve on challenging new tasks, we need a way to infer and invoke the appropriate actions from the policy's rich behavioral prior, especially when directly commanding the policy fails. We focus on flow matching generalists and propose Flow Reversal Steering (FRS): a method that takes suboptimal but ``reasonable'' actions, finds their latent noises by passing them through the flow policy in reverse, and maps them to nearby generalist action modes. We evaluate FRS across many simulated and real-world manipulation settings. First, FRS can turn coarse semantic guidance from humans or vision-language models (VLMs) into corresponding good robot actions, improving zero-shot control. These gains can be distilled with behavioral cloning by training an auxiliary policy to output noises that the generalist maps to good actions -- showing up to 95% absolute task success rate boosts in under a minute of training. Finally, FRS enables policy improvement by bootstrapping reinforcement learning with semantic knowledge, improving on several tasks that standard RL fails to improve on.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. Builds on 61 of its 80 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

61 of 80 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Self-Improving Vision-Language-Action Models with Data Generation via Residual RL30 Oct
  2. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learning19 Sep
  3. 2024Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance17 Oct

Other references · 58

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  6. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  7. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  8. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  9. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  10. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  11. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  12. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
38 more
  1. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  2. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  3. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  4. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  5. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  6. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  7. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 340
  8. 2023Efficient Online Reinforcement Learning with Offline Datacited by 332
  9. 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 256
  10. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 245
  11. 2018Residual Reinforcement Learning for Robot Controlcited by 242
  12. 2022Dual Diffusion Implicit Bridges for Image-to-Image Translationcited by 212
  13. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Modelscited by 201
  14. 2024Taming Rectified Flow for Inversion and Editingcited by 200
  15. 2025LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorizationcited by 166
  16. 2024Semantic Image Inversion and Editing using Rectified Stochastic Differential Equationscited by 162
  17. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learningcited by 153
  18. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
  19. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 142
  20. 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 133
  21. 2025A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulationcited by 125
  22. 2024Stable Flow: Vital Layers for Training-Free Image Editingcited by 111
  23. 2025Reinforcement Learning with Action Chunkingcited by 97
  24. 2026CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulationcited by 72
  25. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 71
  26. 2024FireFlow: Fast Inversion of Rectified Flow for Image Semantic Editingcited by 66
  27. 2024FakeInversion: Learning to Detect Images from Unseen Text-to-Image Models by Inverting Stable Diffusioncited by 61
  28. 2024Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversioncited by 58
  29. 2025RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Modelscited by 56
  30. 2025Diffusion Guidance Is a Controllable Policy Improvement Operatorcited by 51
  31. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 51
  32. 2024CoNVOI: Context-aware Navigation using Vision Language Models in Outdoor and Indoor Environmentscited by 43
  33. 2025UniEdit-Flow: Unleashing Inversion and Editing in the Era of Flow Modelscited by 38
  34. 2024Blind Image Restoration via Fast Diffusion Inversioncited by 29
  35. 2025PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policiescited by 29
  36. 2026TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Roboticscited by 29
  37. 2023Finetuning from Offline Reinforcement Learning: Challenges, Trade-offs and Practical Solutionscited by 26
  38. 2023To the Noise and Back: Diffusion for Shared Autonomycited by 20

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026SAPS: Shared Autonomy for Policy Steering by Blending Teleoperation with a Pretrained VLAcited by 1
  2. 2026HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RLcited by 1
  3. 2026Principal Steering Subspaces for Online Adaptation of Frozen Generative Robot Policiescited by 0
  4. 2026PreferenceFlow: Test-Time Guidance of Flow-Matching Robot Policies from Human Interventionscited by 0
  5. 2026eRLT: Efficient VLA Reinforcement Learning via Action-Relevant Token Routingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Robust Finetuning of Vision-Language-Action Robot Policies via Parameter Merging
  2. 2026Adapting Generalist Robot Policies with Semantic Reinforcement Learning
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Control
  4. 2026VLS: Steering Pretrained Robot Policies via Vision-Language Models
  5. 2024RLDG: Robotic Generalist Policy Distillation via Reinforcement Learning
  6. 2026Proxy Policy Steering ↗
  7. 2026Visual Verification Enables Inference-time Steering and Autonomous Policy Improvement
  8. 2025Reinforcement Learning for Flow-Matching Policies