# Utilizing Reinforcement Learning for de novo Drug Design

arXiv: https://arxiv.org/abs/2303.17615 · HTML: https://mlsanity.com/paper/2303.17615/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 46 references that are in the map.
- First posted on arXiv 30 Mar 2023.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation](https://mlsanity.com/paper/1806.02473/) (2018)
- [GuacaMol: Benchmarking Models for De Novo Molecular Design](https://mlsanity.com/paper/1811.09621/) (2018)
- [Molecular De Novo Design through Deep Reinforcement Learning](https://mlsanity.com/paper/1704.07555/) (2017)
- [Optimization of Molecules via Deep Reinforcement Learning](https://mlsanity.com/paper/1810.08678/) (2018)
- [Sample Efficient Actor-Critic with Experience Replay](https://mlsanity.com/paper/1611.01224/) (2016)
- [Revisiting Fundamentals of Experience Replay](https://mlsanity.com/paper/2007.06700/) (2020)
- [Soft Actor-Critic for Discrete Action Settings](https://mlsanity.com/paper/1910.07207/) (2019)
- [Sample Efficiency Matters: A Benchmark for Practical Molecular Optimization](https://mlsanity.com/paper/2206.12411/) (2022)
- [Local Latent Space Bayesian Optimization over Structured Inputs](https://mlsanity.com/paper/2201.11872/) (2022)
- [Hit and Lead Discovery with Explorative RL and Fragment-based Molecule Generation](https://mlsanity.com/paper/2110.01219/) (2021)
- [Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement Learning](https://mlsanity.com/paper/2004.12485/) (2020)
- [Molecular Design in Synthetically Accessible Chemical Space via Deep Reinforcement Learning](https://mlsanity.com/paper/2004.14308/) (2020)
- [Barking up the right tree: an approach to search over molecule synthesis DAGs](https://mlsanity.com/paper/2012.11522/) (2020)
- [Revisiting Discrete Soft Actor-Critic](https://mlsanity.com/paper/2209.10081/) (2022)
- [Re-evaluating sample efficiency in de novo molecule generation](https://mlsanity.com/paper/2212.01385/) (2022)

## Built on by (most cited first, 7 of 7)

- [3DMolFormer: A Dual-channel Framework for Structure-based Drug Discovery](https://mlsanity.com/paper/2502.05107/) (2025)
- [Tactical Decision Making for Autonomous Trucks by Deep Reinforcement Learning with Total Cost of Operation Based Reward](https://mlsanity.com/paper/2403.06524/) (2024)
- [Diversity-Aware Reinforcement Learning for de novo Drug Design](https://mlsanity.com/paper/2410.10431/) (2024)
- [Diverse Mini-Batch Selection in Reinforcement Learning for Efficient Chemical Exploration in de novo Drug Design](https://mlsanity.com/paper/2506.21158/) (2025)
- [TransDiffSBDD: Causality-Aware Multi-Modal Structure-Based Drug Design](https://mlsanity.com/paper/2503.20913/) (2025)
- [ReACT-Drug: Reaction-Template Guided Reinforcement Learning for de novo Drug Design](https://mlsanity.com/paper/2512.20958/) (2025)
- [Fairness Definitions and Metrics in Deep Reinforcement Learning for Drug Discovery in Healthcare: A Rapid Evidence Review](https://mlsanity.com/paper/2606.02902/) (2026)

## Abstract (arXiv)

Deep learning-based approaches for generating novel drug molecules with specific properties have gained a lot of interest in the last few years. Recent studies have demonstrated promising performance for string-based generation of novel molecules utilizing reinforcement learning. In this paper, we develop a unified framework for using reinforcement learning for de novo drug design, wherein we systematically study various on- and off-policy reinforcement learning algorithms and replay buffers to learn an RNN-based policy to generate novel molecules predicted to be active against the dopamine receptor DRD2. Our findings suggest that it is advantageous to use at least both top-scoring and low-scoring molecules for updating the policy when structural diversity is essential. Using all generated molecules at an iteration seems to enhance performance stability for on-policy algorithms. In addition, when replaying high, intermediate, and low-scoring molecules, off-policy algorithms display the potential of improving the structural diversity and number of active molecules generated, but possibly at the cost of a longer exploration phase. Our work provides an open-source framework enabling researchers to investigate various reinforcement learning methods for de novo drug design.