mlsanityData to 7 Oct 2026

Flow Policy Gradients for Robot Control

arXiv 2602.02481 ↗First posted 2 Feb 2026cs.RO cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Likelihood-based policy gradient methods are the dominant approach for training robot control policies from rewards. These methods rely on differentiable action likelihoods, which constrain policy outputs to simple distributions like Gaussians. In this work, we show how flow matching policy gradients -- a recent framework that bypasses likelihood computation -- can be made effective for training and fine-tuning more expressive policies in challenging robot control settings. We introduce an improved objective that enables success in legged locomotion, humanoid motion tracking, and manipulation tasks, as well as robust sim-to-real transfer on two humanoid robots. We then present ablations and analysis on training dynamics. Results show how policies can exploit the flow representation for exploration when training from scratch, as well as improved fine-tuning robustness over baselines.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 57 of its 63 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

57 of 63 references.

References · 57

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  8. 2016Density estimation using Real NVPcited by 2,029
  9. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  10. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  11. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  12. 2023Training Diffusion Models with Reinforcement Learningcited by 939
38 more
  1. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  2. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  3. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  4. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  5. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
  6. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
  7. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  8. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
  9. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  10. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 293
  11. 2024Diffusion Policy Policy Optimizationcited by 255
  12. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 239
  13. 2021Robust Motion In-betweeningcited by 223
  14. 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 198
  15. 2025Flow Q-Learningcited by 162
  16. 2023Efficient Diffusion Policies for Offline Reinforcement Learningcited by 151
  17. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learningcited by 145
  18. 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 128
  19. 2023Learning a Diffusion Model Policy from Rewards via Q-Score Matchingcited by 126
  20. 2025Training Agents Inside of Scalable World Modelscited by 122
  21. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 104
  22. 2024Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimizationcited by 99
  23. 2023Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learningcited by 93
  24. 2023General In-Hand Object Rotation with Vision and Touchcited by 87
  25. 2025Flow Matching Policy Gradientscited by 86
  26. 2025Adjoint Sampling: Highly Scalable Diffusion Samplers via Adjoint Matchingcited by 78
  27. 2025Energy-Weighted Flow Matching for Offline Reinforcement Learningcited by 70
  28. 2025MuJoCo Playgroundcited by 57
  29. 2025Visual Imitation Enables Contextual Humanoid Controlcited by 57
  30. 2025DIME:Diffusion-Based Maximum Entropy Reinforcement Learningcited by 52
  31. 2025Residual Off-Policy RL for Finetuning Behavior Cloning Policiescited by 42
  32. 2023Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learningcited by 40
  33. 2024The Superposition of Diffusion Models Using the Ito Density Estimatorcited by 36
  34. 2025Much Ado About Noising: Dispelling the Myths of Generative Robotic Controlcited by 35
  35. 2026Q-learning with Adjoint Matchingcited by 33
  36. 2025FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Controlcited by 32
  37. 2024Simple Policy Optimizationcited by 31
  38. 2024Entropy-regularized Diffusion Policy with Q-Ensembles for Offline Reinforcement Learningcited by 29

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

  1. 2026SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-Based Humanoid Controlcited by 4
  2. 2026FlowRL: A Taxonomy and Modular Framework for Reinforcement Learning with Diffusion Policiescited by 2
  3. 2026Scaling Behavior Foundation Model for Humanoid Robotscited by 2
  4. 2026ExpertGen: Scalable Sim-to-Real Expert Policy Learning from Imperfect Behavior Priorscited by 1
  5. 2026Positive-Only Drifting Policy Optimizationcited by 1
  6. 2026OGPO: Sample Efficient Full-Finetuning of Generative Control Policiescited by 1
  7. 2026Preserving Foundational Capabilities in Flow-Matching VLAs through Conservative SFTcited by 1
  8. 2026GenPO++: Generative Policy Optimization with Jacobian-free Likelihood Ratioscited by 1
  9. 2026DiPOD: Diffusion Policy Optimization without Drifting Apartcited by 0
  10. 2026LYRIC: Language-Driven Physics-Based Character Control for Contact-Rich Whole-Body Object Interactioncited by 0
  11. 2026Latent evolving World Action Modelcited by 0
  12. 2026Timestep Weighting: A Hidden Key to Effective ELBO-Based Flow-Matching RLcited by 0
2 more
  1. 2026Constrained Flow Policy Updates: A Generalized Schr"odinger Bridge Viewcited by 0
  2. 2026DexPolicy: Scheduled Exploration for Trajectory-Guided Dexterous Manipulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Extremum Flow Matching for Offline Goal Conditioned Reinforcement Learning
  2. 2026BayesFP: Posterior Estimation for Flow-Based Policies via Feynman-Kac Sampling
  3. 2025Dense-Jump Flow Matching with Non-Uniform Time Scheduling for Robotic Policies: Mitigating Multi-Step Inference Degradation
  4. 2025ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learning
  5. 2026FlowDPG: Deterministic Policy Gradient on Flow Matching Policies for Real-World Manipulation
  6. 2025Reinforcement Learning for Flow-Matching Policies
  7. 2026RoMAN-Flow: Taming Autoregressive Normalizing Flows for Offline Reinforcement Learning in Robotic Manipulation
  8. 2026QF3: Fast Flow RL with Filtered Q-Gradients