# Guiding Diffusion Models with Reinforcement Learning for Stable Molecule Generation

arXiv: https://arxiv.org/abs/2508.16521 · HTML: https://mlsanity.com/paper/2508.16521/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 26 of its 32 references that are in the map.
- First posted on arXiv 22 Aug 2025.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)
- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 26 of 26)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Training Diffusion Models with Reinforcement Learning](https://mlsanity.com/paper/2305.13301/) (2023)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [Equivariant Diffusion for Molecule Generation in 3D](https://mlsanity.com/paper/2203.17003/) (2022)
- [GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation](https://mlsanity.com/paper/2203.02923/) (2022)
- [Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs](https://mlsanity.com/paper/2206.11990/) (2022)
- [Cormorant: Covariant Molecular Neural Networks](https://mlsanity.com/paper/1906.04015/) (2019)
- [Torsional Diffusion for Molecular Conformer Generation](https://mlsanity.com/paper/2206.01729/) (2022)
- [GEOM: Energy-annotated molecular conformations for property prediction and molecular generation](https://mlsanity.com/paper/2006.05531/) (2020)
- [Geometric Latent Diffusion Models for 3D Molecule Generation](https://mlsanity.com/paper/2305.01140/) (2023)
- [Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules](https://mlsanity.com/paper/1906.00957/) (2019)
- [E(n) Equivariant Normalizing Flows](https://mlsanity.com/paper/2105.09016/) (2021)
- [Equivariant 3D-Conditional Diffusion Models for Molecular Linker Design](https://mlsanity.com/paper/2210.05274/) (2022)
- [Diffusion-based Molecule Generation with Informative Prior Bridges](https://mlsanity.com/paper/2209.00865/) (2022)
- [Top-N: Equivariant set and graph generation without exchangeability](https://mlsanity.com/paper/2110.02096/) (2021)
- [UniGEM: A Unified Approach to Generation and Property Prediction for Molecules](https://mlsanity.com/paper/2410.10516/) (2024)
- [Unified Generative Modeling of 3D Molecules via Bayesian Flow Networks](https://mlsanity.com/paper/2403.15441/) (2024)
- [Symphony: Symmetry-Equivariant Point-Centered Spherical Harmonics for 3D Molecule Generation](https://mlsanity.com/paper/2311.16199/) (2023)
- [Graph Diffusion Policy Optimization](https://mlsanity.com/paper/2402.16302/) (2024)
- [Equivariant Neural Diffusion for Molecule Generation](https://mlsanity.com/paper/2506.10532/) (2025)

## Built on by (most cited first, 4 of 4)

- [Elign: Equivariant Diffusion Model Alignment from Foundational Machine Learning Force Fields](https://mlsanity.com/paper/2601.21985/) (2026)
- [RMFlow: Refined Mean Flow by a Noise-Injection Step for Multimodal Generation](https://mlsanity.com/paper/2602.00849/) (2026)
- [MolRGen: A Training and Evaluation Setting for De Novo Molecular Generation with Reasonning Models](https://mlsanity.com/paper/2603.18256/) (2026)
- [PocketVE: Stable and Property-Guided Structure-Based Drug Design with Variance-Exploding Diffusion](https://mlsanity.com/paper/2609.08101/) (2026)

## Abstract (arXiv)

Generating physically realistic 3D molecular structures remains a core challenge in molecular generative modeling. While diffusion models equipped with equivariant neural networks have made progress in capturing molecular geometries, they often struggle to produce equilibrium structures that adhere to physical principles such as force field consistency. To bridge this gap, we propose Reinforcement Learning with Physical Feedback (RLPF), a novel framework that extends Denoising Diffusion Policy Optimization to 3D molecular generation. RLPF formulates the task as a Markov decision process and applies proximal policy optimization to fine-tune equivariant diffusion models. Crucially, RLPF introduces reward functions derived from force-field evaluations, providing direct physical feedback to guide the generation toward energetically stable and physically meaningful structures. Experiments on the QM9 and GEOM-drug datasets demonstrate that RLPF significantly improves molecular stability compared to existing methods. These results highlight the value of incorporating physics-based feedback into generative modeling. The code is available at: https://github.com/ZhijianZhou/RLPF/tree/verl_diffusion.