mlsanityData to 7 Oct 2026

Mean Flow Policy Optimization

arXiv 2604.14698 ↗First posted 16 Apr 2026cs.LGcited by 1

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Diffusion models have recently emerged as expressive policy representations for online reinforcement learning (RL). However, their iterative generative processes introduce substantial training and inference overhead. To overcome this limitation, we propose to represent policies using MeanFlow models, a class of few-step flow-based generative models, to improve training and inference efficiency over diffusion-based RL approaches. To promote exploration, we optimize MeanFlow policies under the maximum entropy RL framework via soft policy iteration, and address two key challenges specific to MeanFlow policies: action likelihood evaluation and soft policy improvement. Experiments on MuJoCo, DeepMind Control Suite and HumanoidBench benchmarks demonstrate that our method, Mean Flow Policy Optimization (MFPO), achieves performance comparable to or exceeding current diffusion-based baselines while considerably reducing training and inference time. Our code is available at https://github.com/dongxiaoyi-xyz/MFPO.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 57 of its 66 references that are in the map. First posted on arXiv 16 Apr 2026.

Builds on

57 of 66 references.

Closest ancestors · 24

Same direction, newest first

  1. 2026Reverse Flow Matching: A Unified Framework for Online Reinforcement Learning with Diffusion and Flow Policies13 Jan
  2. 2025One-Step Flow Policy Mirror Descent31 Jul
  3. 2025ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learning28 May
  4. 2025GenPO: Generative Diffusion Models Meet On-Policy Reinforcement Learning24 May
  5. 2025Energy-Weighted Flow Matching for Offline Reinforcement Learning6 Mar
  6. 2025Maximum Entropy Reinforcement Learning with Diffusion Policy17 Feb
  7. 2025Flow Q-Learning4 Feb
  8. 2025DIME:Diffusion-Based Maximum Entropy Reinforcement Learning4 Feb
  9. 2025Efficient Online Reinforcement Learning for Diffusion Policy1 Feb
  10. 2024Diffusion Policy Policy Optimization1 Sep
  11. 2024Diffusion-DICE: In-Sample Diffusion Guidance for Offline Reinforcement Learning29 Jul
  12. 2024Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning31 May
12 more
  1. 2024Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimization25 May
  2. 2024Diffusion Actor-Critic with Entropy Regulator24 May
  3. 2023Learning a Diffusion Model Policy from Rewards via Q-Score Matching18 Dec
  4. 2023Boosting Continuous Control with Consistency Policy10 Oct
  5. 2023Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learning29 Sep
  6. 2023Efficient Diffusion Policies for Offline Reinforcement Learning31 May
  7. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learning22 May
  8. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning25 Apr
  9. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies20 Apr
  10. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar
  11. 2023Imitating Human Behaviour with Diffusion Models25 Jan
  12. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning12 Aug

Other references · 33

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2022Flow Matching for Generative Modelingcited by 5,426
  9. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  10. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  11. 2018Neural Ordinary Differential Equationscited by 3,733
  12. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
21 more
  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  4. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  5. 2023Consistency Modelscited by 1,778
  6. 2018DeepMind Control Suitecited by 935
  7. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  8. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  9. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  10. 2023Adversarial Diffusion Distillationcited by 741
  11. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  12. 2025Mean Flows for One-step Generative Modelingcited by 524
  13. 2023Improved Techniques for Training Consistency Modelscited by 430
  14. 2024One Step Diffusion via Shortcut Modelscited by 373
  15. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 177
  16. 2024HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulationcited by 101
  17. 2020Distributional Soft Actor-Critic: Off-Policy Reinforcement Learning for Addressing Value Estimation Errorscited by 80
  18. 2025Flow-Based Policy for Online Reinforcement Learningcited by 33
  19. 2025Scaling Offline RL via Efficient and Expressive Shortcut Modelscited by 27
  20. 2025SAC Flow: Sample-Efficient Reinforcement Learning of Flow-Based Policies via Velocity-Reparameterized Sequential Modelingcited by 27
  21. 2024Maximum Entropy Reinforcement Learning via Energy-Based Normalizing Flowcited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Flowing Faster to Coordinate: One-Step Online Multi-Agent Flow Policiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Score-Based One-step MeanFlow Policy Optimization
  2. 2026FlowRL: A Taxonomy and Modular Framework for Reinforcement Learning with Diffusion Policies
  3. 2026Truncated Rectified Flow Policy for Reinforcement Learning with One-Step Sampling
  4. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Process
  5. 2026Boosting Maximum Entropy Reinforcement Learning via One-Step Flow Matching
  6. 2025Training Diffusion Policies via Prior-Mapping Co-Evolution
  7. 2025One-Step Generative Policies with Q-Learning: A Reformulation of MeanFlow
  8. 2025Flow-Based Single-Step Completion for Efficient and Expressive Policy Learning