# Quantum-inspired Reinforcement Learning for Synthesizable Drug Design

arXiv: https://arxiv.org/abs/2409.09183 · HTML: https://mlsanity.com/paper/2409.09183/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 22 of 22)

- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [MolGAN: An implicit generative model for small molecular graphs](https://mlsanity.com/paper/1805.11973/) (2018)
- [Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation](https://mlsanity.com/paper/1806.02473/) (2018)
- [GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generation](https://mlsanity.com/paper/2001.09382/) (2020)
- [GFlowNet Foundations](https://mlsanity.com/paper/2111.09266/) (2021)
- [Molecular De Novo Design through Deep Reinforcement Learning](https://mlsanity.com/paper/1704.07555/) (2017)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [Optimization of Molecules via Deep Reinforcement Learning](https://mlsanity.com/paper/1810.08678/) (2018)
- [GraphDF: A Discrete Flow Model for Molecular Graph Generation](https://mlsanity.com/paper/2102.01189/) (2021)
- [Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models](https://mlsanity.com/paper/1705.10843/) (2017)
- [Multi-Objective De Novo Drug Design with Conditional Graph Generative Model](https://mlsanity.com/paper/1801.07299/) (2018)
- [GraphEBM: Molecular Graph Generation with Energy-Based Models](https://mlsanity.com/paper/2102.00546/) (2021)
- [ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendations](https://mlsanity.com/paper/1908.01425/) (2019)
- [Augmenting Genetic Algorithms with Deep Neural Networks for Exploring the Chemical Space](https://mlsanity.com/paper/1909.11655/) (2019)
- [Differentiable Scaffolding Tree for Molecular Optimization](https://mlsanity.com/paper/2109.10469/) (2021)
- [Guiding Deep Molecular Optimization with Genetic Exploration](https://mlsanity.com/paper/2007.04897/) (2020)
- [MIMOSA: Multi-constraint Molecule Sampling for Molecule Optimization](https://mlsanity.com/paper/2010.02318/) (2020)
- [Reinforced Genetic Algorithm for Structure-based Drug Design](https://mlsanity.com/paper/2211.16508/) (2022)
- [Amortized Tree Generation for Bottom-up Synthesis Planning and Synthesizable Molecular Design](https://mlsanity.com/paper/2110.06389/) (2021)
- [Deep Molecular Dreaming: Inverse machine learning for de-novo molecular design and interpretability with surjective representations](https://mlsanity.com/paper/2012.09712/) (2020)
- [Uncertainty Quantification on Clinical Trial Outcome Prediction](https://mlsanity.com/paper/2401.03482/) (2024)

## Abstract (arXiv)

Synthesizable molecular design (also known as synthesizable molecular optimization) is a fundamental problem in drug discovery, and involves designing novel molecular structures to improve their properties according to drug-relevant oracle functions (i.e., objective) while ensuring synthetic feasibility. However, existing methods are mostly based on random search. To address this issue, in this paper, we introduce a novel approach using the reinforcement learning method with quantum-inspired simulated annealing policy neural network to navigate the vast discrete space of chemical structures intelligently. Specifically, we employ a deterministic REINFORCE algorithm using policy neural networks to output transitional probability to guide state transitions and local search using genetic algorithm to refine solutions to a local optimum within each iteration. Our methods are evaluated with the Practical Molecular Optimization (PMO) benchmark framework with a 10K query budget. We further showcase the competitive performance of our method by comparing it against the state-of-the-art genetic algorithms-based method.