# RL with diffusion and flow models

Area: [Deep reinforcement learning methods](https://mlsanity.com/deeprl/index.md). HTML: https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/

## Key facts

- RL with diffusion and flow models: 346 papers on arXiv since 2024 (data through 14 Oct 2026).
- 67 of its 346 mapped papers (19%) cite “Diffusion Policy Policy Optimization” (2024, arXiv 2409.00588).
- 51 of its 346 mapped papers (15%) cite “Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimization” (2024, arXiv 2405.16173).
- 16 new papers 30 Sep–14 Oct.

## Most cited here

- [Diffusion Policy Policy Optimization](https://mlsanity.com/paper/2409.00588/) (2024), cited by 67 papers here
- [Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimization](https://mlsanity.com/paper/2405.16173/) (2024), cited by 51 papers here
- [Flow Q-Learning](https://mlsanity.com/paper/2502.02538/) (2025), cited by 48 papers here
- [Diffusion Actor-Critic with Entropy Regulator](https://mlsanity.com/paper/2405.15177/) (2024), cited by 47 papers here
- [Flow Matching Policy Gradients](https://mlsanity.com/paper/2507.21053/) (2025), cited by 35 papers here
- [Efficient Online Reinforcement Learning for Diffusion Policy](https://mlsanity.com/paper/2502.00361/) (2025), cited by 30 papers here
- [ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learning](https://mlsanity.com/paper/2505.22094/) (2025), cited by 30 papers here
- [DIME:Diffusion-Based Maximum Entropy Reinforcement Learning](https://mlsanity.com/paper/2502.02316/) (2025), cited by 28 papers here
- [Learning Multimodal Behaviors from Scratch with Diffusion Policy Gradient](https://mlsanity.com/paper/2406.00681/) (2024), cited by 21 papers here
- [Maximum Entropy Reinforcement Learning with Diffusion Policy](https://mlsanity.com/paper/2502.11612/) (2025), cited by 19 papers here

## New (30 Sep–14 Oct)

- [SAM3RL: Memory Control via Reinforcement Learning for Visual Object Tracking](https://mlsanity.com/paper/2610.19931/) (2026-10-14)
- [PrT-Flow: Probability-Tractable Flow Matching for Reinforcement Learning](https://mlsanity.com/paper/2610.17667/) (2026-10-13)
- [DR-APO: Decoupled-Representation Advantage-Guided Policy Optimization from Continuous Control to Embodied Navigation](https://mlsanity.com/paper/2610.16618/) (2026-10-12)
- [DEFlow: Deep Equilibrium Flow Maximum-Entropy Reinforcement Learning](https://mlsanity.com/paper/2610.16597/) (2026-10-12)
- [XAI-Guided Fourier Domain Generalisation for Pure-Simulation Sim-to-Real Deep Reinforcement Learning](https://mlsanity.com/paper/2610.16478/) (2026-10-12)
- [A Survey of Flow Matching in Reinforcement Learning](https://mlsanity.com/paper/2610.16275/) (2026-10-12)
- [Drift, Then Spread: Learning One-Step Generative Policies for Online Reinforcement Learning](https://mlsanity.com/paper/2610.15662/) (2026-10-11)
- [Single-step Retrosynthesis via Reinforcement Learning from Forward Feedback](https://mlsanity.com/paper/2610.15141/) (2026-10-10)
- [MORAD: Reinforcement Learning with Multi-Objective Rewards for RNA Inverse Design](https://mlsanity.com/paper/2610.13305/) (2026-10-09)
- [Reinforcement Learning via Fixed-point Flow Matching](https://mlsanity.com/paper/2610.12848/) (2026-10-09)
- [Q-Learning with Scalar Adjoint Matching](https://mlsanity.com/paper/2610.10437/) (2026-10-07)
- [Safe Image Generation via Reinforcement Learning](https://mlsanity.com/paper/2610.05908/) (2026-10-05)
- [PWM: Personalized World Models with Online Reinforcement Learning](https://mlsanity.com/paper/2610.04920/) (2026-10-04)
- [Score-Calibrated Flow for Sampling from Unnormalized Densities with Applications to Generative Online Reinforcement Learning](https://mlsanity.com/paper/2610.04696/) (2026-10-03)
- [Reinforcement Learning on the Discrete Composition Channel of a Crystal Generator: Validated Gains and Reward Hacking](https://mlsanity.com/paper/2610.03880/) (2026-10-02)
- [VIGOR: Zero-Shot Visual Generalization via Latent-Space Consistency in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2610.02801/) (2026-10-02)