# Positive-Only Drifting Policy Optimization

arXiv: https://arxiv.org/abs/2604.16519 · HTML: https://mlsanity.com/paper/2604.16519/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 7 of its 11 references that are in the map.
- First posted on arXiv 15 Apr 2026.

## Placed in

- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 7 of 7)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Generative Modeling via Drifting](https://mlsanity.com/paper/2602.04770/) (2026)
- [Flow Matching Policy Gradients](https://mlsanity.com/paper/2507.21053/) (2025)
- [DiffuseLoco: Real-Time Legged Locomotion Control with Diffusion from Offline Datasets](https://mlsanity.com/paper/2404.19264/) (2024)
- [Flow Policy Gradients for Robot Control](https://mlsanity.com/paper/2602.02481/) (2026)
- [Integrating Diffusion-based Multi-task Learning with Online Reinforcement Learning for Robust Quadruped Robot Control](https://mlsanity.com/paper/2507.05674/) (2025)

## Built on by (most cited first, 2 of 2)

- [One-Step Generative Modeling via Unbalanced Optimal Transport](https://mlsanity.com/paper/2609.32708/) (2026)
- [Occupancy Modeling via Drifting: One-Step Generation Enables Direct Policy Extraction](https://mlsanity.com/paper/2610.19883/) (2026)

## Abstract (arXiv)

In the field of online reinforcement learning (RL), traditional Gaussian policies and flow-based methods are often constrained by their unimodal expressiveness, complex gradient clipping, or stringent trust-region requirements. Moreover, they all rely on post-hoc penalization of negative samples to correct erroneous actions. This paper introduces Positive-Only Drifting Policy Optimization (PODPO), a likelihood-free and gradient-clipping-free generative approach for online RL. By leveraging the drifting model, PODPO performs policy updates via advantage-weighted local contrastive drifting. Relying solely on positive-advantage samples, it elegantly steers actions toward high-return regions while exploiting the inherent local smoothness of the generative model to enable proactive error prevention. In doing so, PODPO opens a promising new pathway for generative policy learning in online settings.