mlsanityData to 7 Oct 2026

Is Conditional Generative Modeling all you need for Decision-Making?

arXiv 2211.15657 ↗First posted 28 Nov 2022cs.LG cs.AIcited by 504

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Recent improvements in conditional generative modeling have made it possible to generate high-quality images from language descriptions alone. We investigate whether these methods can directly address the problem of sequential decision-making. We view decision-making not through the lens of reinforcement learning (RL), but rather through conditional generative modeling. To our surprise, we find that our formulation leads to policies that can outperform existing offline RL approaches across standard benchmarks. By modeling a policy as a return-conditional diffusion model, we illustrate how we may circumvent the need for dynamic programming and subsequently eliminate many of the complexities that come with traditional offline RL. We further demonstrate the advantages of modeling policies as conditional diffusion models by considering two other conditioning variables: constraints and skills. Conditioning on a single constraint or skill during training leads to behaviors at test-time that can satisfy several constraints together or demonstrate a composition of skills. Our results illustrate that conditional generative modeling is a powerful tool for decision-making.

Key facts. Cited by 504 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 55 of its 71 references that are in the map. First posted on arXiv 28 Nov 2022.

Builds on

55 of 71 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning12 Aug

Other references · 54

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  12. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
38 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  6. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  8. 2018Group Normalizationcited by 1,514
  9. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  10. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  11. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  12. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  13. 2021Variational Diffusion Modelscited by 1,145
  14. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  15. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  16. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  17. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  18. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  19. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  20. 2022Understanding Diffusion Models: A Unified Perspectivecited by 337
  21. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
  22. 2022Online Decision Transformercited by 194
  23. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 184
  24. 2018Deep Reinforcement Learning and the Deadly Triadcited by 170
  25. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  26. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 152
  27. 2019Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Modelcited by 147
  28. 2019Mish: A Self Regularized Non-Monotonic Activation Functioncited by 141
  29. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 128
  30. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 124
  31. 2018Zero-Shot Visual Imitationcited by 114
  32. 2019Training Agents using Upside-Down Reinforcement Learningcited by 106
  33. 2019Reward-Conditioned Policiescited by 101
  34. 2020Chip Placement with Deep Reinforcement Learningcited by 83
  35. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learningcited by 68
  36. 2022You Can't Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environmentscited by 62
  37. 2021Shaking the foundations: delusions in sequence models for interaction and controlcited by 61
  38. 2022Offline RL Policies Should be Trained to be Adaptivecited by 52

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 88; 2024: 154; 2025: 158; 2026: 104.

504 papers.

Sort:

By area:

  1. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  2. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  3. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  4. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  5. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
  6. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 307
  7. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  8. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
  9. 2023Imitating Human Behaviour with Diffusion Modelscited by 260
  10. 2024Diffusion Policy Policy Optimizationcited by 255
  11. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 246
  12. 2023Goal-Conditioned Imitation Learning using Score-based Diffusion Policiescited by 229
38 more
  1. 2025Real-Time Execution of Action Chunking Flow Policiescited by 199
  2. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
  3. 2025Flow Q-Learningcited by 162
  4. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  5. 2023Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Modelscited by 155
  6. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
  7. 2023AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Plannerscited by 147
  8. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learningcited by 145
  9. 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 138
  10. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  11. 2024THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulationcited by 128
  12. 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 128
  13. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 117
  14. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 115
  15. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
  16. 2023Language-Guided Traffic Simulation via Scene-Level Diffusioncited by 110
  17. 2024GRAPE: Generalizing Robot Policy via Preference Alignmentcited by 106
  18. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learningcited by 105
  19. 2023Synthetic Experience Replaycited by 100
  20. 2023Guided Flows for Generative Modeling and Decision Makingcited by 99
  21. 2024Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimizationcited by 99
  22. 2023Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learningcited by 93
  23. 2024Hierarchical Diffusion Policy for Kinematics-Aware Multi-Task Robotic Manipulationcited by 92
  24. 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Reviewcited by 92
  25. 2023Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Modelscited by 87
  26. 2025Flow Matching Policy Gradientscited by 86
  27. 2023Diffusion Models for Reinforcement Learning: A Surveycited by 84
  28. 2024One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillationcited by 83
  29. 2024Diffusion Actor-Critic with Entropy Regulatorcited by 81
  30. 2024From Imitation to Refinement -- Residual RL for Precise Assemblycited by 80
  31. 2023MADiff: Offline Multi-agent Learning with Diffusion Modelscited by 73
  32. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  33. 2024Simple Hierarchical Planning with Diffusioncited by 71
  34. 2024Amortizing intractable inference in diffusion models for vision, language, and controlcited by 70
  35. 2025Energy-Weighted Flow Matching for Offline Reinforcement Learningcited by 70
  36. 2023SafeDiffuser: Safe Planning with Diffusion Probabilistic Modelscited by 69
  37. 2023SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Executioncited by 69
  38. 2023Score Regularized Policy Optimization through Diffusion Behaviorcited by 67

50 most cited of 504 All 504 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Revisiting Generative Policies: A Simpler Reinforcement Learning Algorithmic Perspective
  2. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modeling
  3. 2023Reinforcement Learning for Generative AI: A Survey
  4. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directions
  5. 2025Offline Reinforcement Learning with Generative Trajectory Policies
  6. 2026ParetoSlider: Diffusion Models Post-Training for Continuous Reward Control
  7. 2024Adding Conditional Control to Diffusion Models with Reinforcement Learning
  8. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL