mlsanityData to 7 Oct 2026

Diffusion and flow policies for offline RL

155 papers3 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: diffusion, offline, policy, reinforcement learning, reinforcement

Key facts. Diffusion and flow policies for offline RL: 155 papers on arXiv since 2022 (data through 7 Oct 2026). 47 of its 155 mapped papers (30%) cite “Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning” (2022, arXiv 2208.06193). 34 of its 155 mapped papers (22%) cite “IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies” (2023, arXiv 2304.10573). 12 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

12 new papers, 17 Sep – 7 Oct; 3 this week.

  1. This week · 3
  2. World-Model Policy Arbiter for Goal-Conditioned Reinforcement Learning also fits: Offline RL algorithms
  3. LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL also fits: Offline RL algorithms
  4. Variance-Averse n-Step Offline Reinforcement Learning for Sparse Long-Horizon Environments also fits: Offline RL algorithms
  5. Earlier · 9
  6. Optimal Transport Meets Reinforcement Learning: A Survey also fits: Offline RL algorithms
  7. Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning also fits: Offline RL algorithms
  8. Diffusion Policy Improvement with Proposal-Conditioned Refinement Flows
  9. HorizonFlow: Variable-Length Planning for Offline Goal-Conditioned RL
  10. Diffusion Subgoal Planning for Long-Horizon Offline Goal-Conditioned Reinforcement Learning
  11. QAMM: Adjoint MeanFlow Matching for Few-Step Offline Reinforcement Learning
  12. GTRL: Grounding Divide-and-Conquer Value Learning with Temporal Differences also fits: Offline RL algorithms
All 12
  1. Schrödinger--Föllmer Actor--Critic: Diffusion Policy Improvement with Finite-Sample Analysis
  2. Retimed Bellman Flows: Escaping the Impossible Triangle of Velocity Bootstrapping

Foundations

Most cited by papers here

  1. Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning2022cited by 47 here
  2. IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies2023cited by 34 here
  3. Is Conditional Generative Modeling all you need for Decision-Making?2022cited by 28 here
  4. Offline Reinforcement Learning via High-Fidelity Generative Behavior Modeling2022cited by 25 here
  5. Efficient Diffusion Policies for Offline Reinforcement Learning2023cited by 24 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2019Behavior Regularized Offline Reinforcement Learningcited by 35 here
  2. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 59 here
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 45 here
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 63 here
  5. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 22 here
  6. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 46 here
  7. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 60 here
  8. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 47 here
  9. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 25 here
  10. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learningcited by 19 here
14 more it builds on, not in this chain
  1. 2019Learning to Reach Goals via Iterated Supervised Learningcited by 20 here
  2. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 14 here
  3. 2020Critic Regularized Regressioncited by 16 here
  4. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 14 here
  5. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 18 here
  6. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RLcited by 18 here
  7. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 28 here
  8. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 34 here
  9. 2023Efficient Diffusion Policies for Offline Reinforcement Learningcited by 24 here
  10. 2023Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learningcited by 14 here
  11. 2023Score Regularized Policy Optimization through Diffusion Behaviorcited by 15 here
  12. 2024Diffusion Policies creating a Trust Region for Offline Reinforcement Learningcited by 13 here
  13. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 22 here
  14. 2025Flow Q-Learningcited by 19 here

The 24 most cited of 389.

Browse

30 of 155, ranked 6–35

  1. 6OGBench: Benchmarking Offline Goal-Conditioned RL202422
  2. 7Flow Q-Learning202519
  3. 8Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning202319
  4. 9Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL202218
  5. 10Score Regularized Policy Optimization through Diffusion Behavior202315
  6. 11Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learning202314
  7. 12Diffusion Policies creating a Trust Region for Offline Reinforcement Learning202413
  8. 13AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners202311
  9. 14How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regression202210
  10. 15Energy-Weighted Flow Matching for Offline Reinforcement Learning20257
All 30
  1. 16Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning20247
  2. 17Diffusion-DICE: In-Sample Diffusion Guidance for Offline Reinforcement Learning20247
  3. 18Entropy-regularized Diffusion Policy with Q-Ensembles for Offline Reinforcement Learning20246
  4. 19DiffCPS: Diffusion Model based Constrained Policy Search for Offline Reinforcement Learning20236
  5. 20Learning a Diffusion Model Policy from Rewards via Q-Score Matching20236
  6. 21Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learning20226
  7. 22What Makes a Good Diffusion Planner for Decision Making?20255
  8. 23Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learning20255
  9. 24MetaDiffuser: Diffusion Model as Conditional Planner for Offline Meta-RL20235
  10. 25Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learning20235
  11. 26What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?20234
  12. 27Physics-informed Value Learner for Offline Goal-Conditioned Reinforcement Learning20253
  13. 28Goal-Conditioned Imitation Learning using Score-based Diffusion Policies20233
  14. 29Instructed Diffuser with Temporal Condition Guidance for Offline Reinforcement Learning20233
  15. 30SMORE: Score Models for Offline Goal-Conditioned Reinforcement Learning20233
  16. 31Behavior-Regularized Diffusion Policy Optimization for Offline Reinforcement Learning20252
  17. 32Habitizing Diffusion Planning for Efficient and Effective Decision Making20252
  18. 33Accelerating Diffusion Planners in Offline RL via Reward-Aware Consistency Trajectory Distillation20252
  19. 34Offline Goal-conditioned Reinforcement Learning with Quasimetric Representations20252
  20. 35One-Step Generative Policies with Q-Learning: A Reformulation of MeanFlow20252