# Diffusion and flow policies for offline RL

Area: [Offline RL and imitation learning from data](https://mlsanity.com/offlinerl/index.md). HTML: https://mlsanity.com/offlinerl/diffusion-and-flow-policies-for-offline-rl/

## Key facts

- Diffusion and flow policies for offline RL: 155 papers on arXiv since 2022 (data through 14 Oct 2026).
- 47 of its 155 mapped papers (30%) cite “Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning” (2022, arXiv 2208.06193).
- 34 of its 155 mapped papers (22%) cite “IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies” (2023, arXiv 2304.10573).
- 16 new papers 17 Sep–14 Oct.

## Most cited here

- [Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning](https://mlsanity.com/paper/2208.06193/) (2022), cited by 47 papers here
- [IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies](https://mlsanity.com/paper/2304.10573/) (2023), cited by 34 papers here
- [Is Conditional Generative Modeling all you need for Decision-Making?](https://mlsanity.com/paper/2211.15657/) (2022), cited by 28 papers here
- [Offline Reinforcement Learning via High-Fidelity Generative Behavior Modeling](https://mlsanity.com/paper/2209.14548/) (2022), cited by 25 papers here
- [Efficient Diffusion Policies for Offline Reinforcement Learning](https://mlsanity.com/paper/2305.20081/) (2023), cited by 24 papers here
- [OGBench: Benchmarking Offline Goal-Conditioned RL](https://mlsanity.com/paper/2410.20092/) (2024), cited by 22 papers here
- [Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning](https://mlsanity.com/paper/2304.12824/) (2023), cited by 19 papers here
- [Flow Q-Learning](https://mlsanity.com/paper/2502.02538/) (2025), cited by 19 papers here
- [Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL](https://mlsanity.com/paper/2202.04478/) (2022), cited by 18 papers here
- [Score Regularized Policy Optimization through Diffusion Behavior](https://mlsanity.com/paper/2310.07297/) (2023), cited by 15 papers here

## New (17 Sep–14 Oct)

- [Value-Guided Repulsion for Offline Reinforcement Learning](https://mlsanity.com/paper/2610.20190/) (2026-10-14)
- [From Suboptimal to Optimal: One-Step Latent Flow Policies for Offline Reinforcement Learning](https://mlsanity.com/paper/2610.16984/) (2026-10-12)
- [Compositional Average Velocity Modeling for Efficient Diffusion Offline Reinforcement Learning](https://mlsanity.com/paper/2610.14941/) (2026-10-10)
- [CFG-Policy: Unified Diffusion Policy Learning Framework via Classifier-free Guidance](https://mlsanity.com/paper/2610.14936/) (2026-10-10)
- [World-Model Policy Arbiter for Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2610.10932/) (2026-10-07)
- [LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL](https://mlsanity.com/paper/2610.08989/) (2026-10-06)
- [Variance-Averse n-Step Offline Reinforcement Learning for Sparse Long-Horizon Environments](https://mlsanity.com/paper/2610.07899/) (2026-10-06)
- [Optimal Transport Meets Reinforcement Learning: A Survey](https://mlsanity.com/paper/2610.01413/) (2026-10-01)
- [Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2610.00849/) (2026-10-01)
- [Diffusion Policy Improvement with Proposal-Conditioned Refinement Flows](https://mlsanity.com/paper/2609.36812/) (2026-10-01)
- [HorizonFlow: Variable-Length Planning for Offline Goal-Conditioned RL](https://mlsanity.com/paper/2609.36896/) (2026-09-29)
- [Diffusion Subgoal Planning for Long-Horizon Offline Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2609.34575/) (2026-09-28)
- [QAMM: Adjoint MeanFlow Matching for Few-Step Offline Reinforcement Learning](https://mlsanity.com/paper/2609.34497/) (2026-09-28)
- [GTRL: Grounding Divide-and-Conquer Value Learning with Temporal Differences](https://mlsanity.com/paper/2609.33259/) (2026-09-27)
- [Schrödinger--Föllmer Actor--Critic: Diffusion Policy Improvement with Finite-Sample Analysis](https://mlsanity.com/paper/2609.33239/) (2026-09-27)
- [Retimed Bellman Flows: Escaping the Impossible Triangle of Velocity Bootstrapping](https://mlsanity.com/paper/2609.32828/) (2026-09-26)