mlsanityData to 14 Oct 2026

Graph Diffusion Policy Optimization

arXiv 2402.16302 ↗First posted 26 Feb 2024cs.LG cs.AI cs.CEcited by 13

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Recent research has made significant progress in optimizing diffusion models for downstream objectives, which is an important pursuit in fields such as graph generation for drug design. However, directly applying these models to graph presents challenges, resulting in suboptimal performance. This paper introduces graph diffusion policy optimization (GDPO), a novel approach to optimize graph diffusion models for arbitrary (e.g., non-differentiable) objectives using reinforcement learning. GDPO is based on an eager policy gradient tailored for graph diffusion models, developed through meticulous analysis and promising improved performance. Experimental results show that GDPO achieves state-of-the-art performance in various graph generation tasks with complex and diverse objectives. Code is available at https://github.com/sail-sg/GDPO.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 50 of its 77 references that are in the map. First posted on arXiv 26 Feb 2024.

Builds on

50 of 77 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagation5 Oct
  2. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewards29 Sep
  3. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models25 May
  4. 2023Training Diffusion Models with Reinforcement Learning22 May

Other references · 46

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  9. 2016Variational Graph Auto-Encoderscited by 1,382
  10. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 520
  11. 2023Aligning Text-to-Image Models using Human Feedbackcited by 422
  12. 2018GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Modelscited by 413
34 more
  1. 2018MolGAN: An implicit generative model for small molecular graphscited by 393
  2. 2018GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoderscited by 383
  3. 2018Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generationcited by 364
  4. 2020GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generationcited by 305
  5. 2022Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equationscited by 283
  6. 2018Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Modelscited by 275
  7. 2020Permutation Invariant Graph Generation via Score-Based Generative Modelingcited by 270
  8. 2017Molecular De Novo Design through Deep Reinforcement Learningcited by 267
  9. 2016Molecular Graph Convolutions: Moving Beyond Fingerprintscited by 238
  10. 2019Efficient Graph Generation with Graph Recurrent Attention Networkscited by 234
  11. 2020Hierarchical Generation of Molecular Graphs using Structural Motifscited by 208
  12. 2018Optimization of Molecules via Deep Reinforcement Learningcited by 200
  13. 2018Constrained Graph Variational Autoencoders for Molecule Designcited by 185
  14. 2021GraphDF: A Discrete Flow Model for Molecular Graph Generationcited by 180
  15. 2021MARS: Markov Molecular Sampling for Multi-objective Drug Discoverycited by 125
  16. 2017GraphGAN: Graph Representation Learning with Generative Adversarial Netscited by 121
  17. 2018Graphite: Iterative Generative Modeling of Graphscited by 112
  18. 2019Graph Normalizing Flowscited by 104
  19. 2019Exploring Randomly Wired Neural Networks for Image Recognitioncited by 103
  20. 2022SPECTRE: Spectral Conditioning Helps to Overcome the Expressivity Limits of One-shot Graph Generatorscited by 99
  21. 2022Exploring Chemical Space with Score-based Out-of-distribution Generationcited by 95
  22. 2020A Systematic Survey on Deep Generative Models for Graph Generationcited by 94
  23. 2023Optimizing DDPM Sampling with Shortcut Fine-Tuningcited by 81
  24. 2017Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedbackcited by 72
  25. 2021Hit and Lead Discovery with Explorative RL and Fragment-based Molecule Generationcited by 68
  26. 2018Generative Code Modeling with Graphscited by 60
  27. 2023Graph Generation with Diffusion Mixturecited by 55
  28. 2019Semi-Implicit Graph Variational Auto-Encoderscited by 54
  29. 2022LIMO: Latent Inceptionism for Targeted Molecule Generationcited by 40
  30. 2023A Survey on Graph Diffusion Models: Generative AI in Science for Molecule, Protein and Materialcited by 35
  31. 2022MGCVAE: Multi-objective Inverse Design via Molecular Graph Conditional Variational Autoencodercited by 16
  32. 2020We Should at Least Be Able to Design Molecules That Dock Wellcited by 9
  33. 2020Conditional Constrained Graph Variational Autoencoders for Molecule Designcited by 6
  34. 2021Spatial Graph Attention and Curiosity-driven Policy for Antiviral Drug Discoverycited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 4.

13 papers.

Sort:

By area:

  1. 2024Improving Molecular Graph Generation with Flow Matching and Optimal Transportcited by 10
  2. 2025Multi-agent Coordination via Flow Matchingcited by 10
  3. 2025Diffusion Models for Future Networks and Communications: A Comprehensive Surveycited by 6
  4. 2026Graph-GRPO: Training Graph Flow Models with Reinforcement Learningcited by 5
  5. 2025Guiding Diffusion Models with Reinforcement Learning for Stable Molecule Generationcited by 4
  6. 2025Fast Graph Generation via Autoregressive Noisy Filtration Modelingcited by 2
  7. 2025DMol: A Highly Efficient and Chemical Motif-Preserving Molecule Generation Platformcited by 1
  8. 2025Topology Generation of UAV Covert Communication Networks: A Graph Diffusion Approach with Incentive Mechanismcited by 0
  9. 2025Cognitive Structure Generation: From Educational Priors to Policy Optimizationcited by 0
  10. 2025CoPHo: Classifier-guided Conditional Topology Generation with Persistent Homologycited by 0
  11. 2026DGPO: RL-Steered Graph Diffusion for Neural Architecture Generationcited by 0
  12. 2026Graph Convolutional Attention: A Spectral Perspective on Graph Denoising and Diffusioncited by 0
1 more
  1. 2026Learning Graph Energies for Finite-Time Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Diffusion Policy Policy Optimization
  2. 2025Graph Guided Diffusion: Unified Guidance for Conditional Graph Generation
  3. 2024Graph Reinforcement Learning for Combinatorial Optimization: A Survey and Unifying Perspective
  4. 2026iADD: Improving Alignment and Diversity in Diffusion Policy Optimization
  5. 2024Discrete-state Continuous-time Diffusion for Graph Generation
  6. 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Review
  7. 2026Explore or Converge? Stage-Guided Per-Step Optimization for Diffusion Models
  8. 2025Fine-Tuning Discrete Diffusion Models with Policy Gradient Methods