# PGFS++: Molecular Property Improvement under Synthesis and Diversity Constraints

arXiv: https://arxiv.org/abs/2608.19121 · HTML: https://mlsanity.com/paper/2608.19121/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 23 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 9 of 9)

- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [MolGAN: An implicit generative model for small molecular graphs](https://mlsanity.com/paper/1805.11973/) (2018)
- [Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation](https://mlsanity.com/paper/1806.02473/) (2018)
- [Augmenting Genetic Algorithms with Deep Neural Networks for Exploring the Chemical Space](https://mlsanity.com/paper/1909.11655/) (2019)
- [Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement Learning](https://mlsanity.com/paper/2004.12485/) (2020)
- [Reinforcement Learning for Molecular Design Guided by Quantum Mechanics](https://mlsanity.com/paper/2002.07717/) (2020)
- [Genetic algorithms are strong baselines for molecule generation](https://mlsanity.com/paper/2310.09267/) (2023)
- [SynFlowNet: Design of Diverse and Novel Molecules with Synthesis Constraints](https://mlsanity.com/paper/2405.01155/) (2024)
- [Tanimoto Random Features for Scalable Molecular Machine Learning](https://mlsanity.com/paper/2306.14809/) (2023)

## Abstract (arXiv)

Improving molecular properties, such as drug-likeness or binding affinity, is a recurring task in early-stage drug discovery. However, molecules optimized in an unconstrained chemical space have limited practical value if they cannot be synthesized. Policy Gradient for Forward Synthesis (PGFS) is a synthesis-aware reinforcement learning method for molecular improvement, but its use of reactant embedding prediction makes reactant selection indirect, which, as we show, limits learning effectiveness. We first develop PGFS+, in which reaction templates and second reactants are represented by trainable embedding lookup tables. Combined with a more effective scoring function and RL algorithm, PGFS+ significantly improves the desired property. However, it exposes a reward-hacking failure mode: a powerful reactant search can map diverse input molecules to the same high-reward magnet molecule, improving the reward while collapsing the output diversity. We therefore introduce PGFS++, a synthesis-aware reinforcement learning framework for input-specific molecular improvement. Given an input molecule, PGFS++ treats it as the start of a forward-synthesis trajectory, applies learned reaction templates with compatible in-stock building blocks, and produces a molecule with improved target properties, an explicit synthesis route, and structural similarity to the input. Experiments on molecular improvement tasks show that PGFS++ improves target properties while preserving high output diversity.