mlsanityData to 14 Oct 2026

Drifting Field Policy: A One-Step Generative Policy via Wasserstein Gradient Flow

arXiv 2605.07727 ↗First posted 8 May 2026cs.LG cs.AI cs.ROcited by 3

Not on the map. Why?

Abstract (arXiv)

We propose Drifting Field Policy (DFP), a non-ODE one-step generative policy built on the drifting model paradigm. We frame the policy update as a reverse-KL Wasserstein-2 gradient flow toward a soft target policy, so that each DFP update corresponds to a gradient step in probability space. By construction, this gradient is decomposed into an ascent toward higher action-value regions and a score matching with the anchor policy as a trust region. We further derive a simple, tractable surrogate of the otherwise intractable update loss, akin to behavior cloning on top-K critic-selected actions. We find empirically that this mechanism uniquely benefits the drifting backbone owing to its non-ODE parameterization. With one-step inference, DFP achieves state-of-the-art performance on several manipulation tasks across Robomimic and OGBench, outperforming ODE-based policies.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 56 of its 66 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

56 of 66 references.

References · 56

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  7. 2015Continuous control with deep reinforcement learningcited by 3,407
  8. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  9. 2015Trust Region Policy Optimizationcited by 2,580
  10. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  11. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  12. 2023Consistency Modelscited by 1,822
38 more
  1. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  2. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  3. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  4. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  5. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  6. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  7. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  8. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 671
  9. 2019Behavior Regularized Offline Reinforcement Learningcited by 662
  10. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  11. 2025Mean Flows for One-step Generative Modelingcited by 561
  12. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  13. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusioncited by 459
  14. 2024One Step Diffusion via Shortcut Modelscited by 391
  15. 2023Efficient Online Reinforcement Learning with Offline Datacited by 363
  16. 2018Maximum a Posteriori Policy Optimisationcited by 328
  17. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 325
  18. 2024Diffusion Policy Policy Optimizationcited by 280
  19. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 238
  20. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuningcited by 237
  21. 2025Flow Q-Learningcited by 184
  22. 2024Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillationcited by 166
  23. 2022Hybrid RL: Using Both Offline and Online Data Can Make RL Efficientcited by 161
  24. 2023Revisiting the Minimalist Approach to Offline Reinforcement Learningcited by 126
  25. 2026Generative Modeling via Driftingcited by 116
  26. 2020EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RLcited by 115
  27. 2024Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimizationcited by 110
  28. 2025Reinforcement Learning with Action Chunkingcited by 107
  29. 2023Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learningcited by 100
  30. 2025Flow Matching Policy Gradientscited by 99
  31. 2023Score Regularized Policy Optimization through Diffusion Behaviorcited by 71
  32. 2025Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcingcited by 38
  33. 2026Q-learning with Adjoint Matchingcited by 37
  34. 2025MP1: MeanFlow Tames Policy Learning in 1-step for Robotic Manipulationcited by 35
  35. 2025Scaling Offline RL via Efficient and Expressive Shortcut Modelscited by 27
  36. 2026Sinkhorn-Drifting Generative Modelscited by 24
  37. 2026Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspectivecited by 20
  38. 2025One-Step Generative Policies with Q-Learning: A Reformulation of MeanFlowcited by 18

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026CoDrift: Compositional Drifting for Offline Reinforcement Learningcited by 0
  2. 2026Drift, Then Spread: Learning One-Step Generative Policies for Online Reinforcement Learningcited by 0
  3. 2026Occupancy Modeling via Drifting: One-Step Generation Enables Direct Policy Extractioncited by 0