# Curiosity as a Self-Supervised Method to Improve Exploration in De novo Drug Design

arXiv: https://arxiv.org/abs/2401.06771 · HTML: https://mlsanity.com/paper/2401.06771/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 33 references that are in the map.
- First posted on arXiv 24 Sep 2023.

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models](https://mlsanity.com/paper/1811.12823/) (2018)
- [Molecular De Novo Design through Deep Reinforcement Learning](https://mlsanity.com/paper/1704.07555/) (2017)
- [Optimization of Molecules via Deep Reinforcement Learning](https://mlsanity.com/paper/1810.08678/) (2018)
- [Deep Reinforcement Learning for De-Novo Drug Design](https://mlsanity.com/paper/1711.10907/) (2017)
- [Learning to Utilize Shaping Rewards: A New Approach of Reward Shaping](https://mlsanity.com/paper/2011.02669/) (2020)

## Built on by (most cited first, 1 of 1)

- [Mol-AIR: Molecular Reinforcement Learning with Adaptive Intrinsic Rewards for Goal-directed Molecular Generation](https://mlsanity.com/paper/2403.20109/) (2024)

## Abstract (arXiv)

In recent years, deep learning has demonstrated promising results in de novo drug design. However, the proposed techniques still lack an efficient exploration of the large chemical space. Most of these methods explore a small fragment of the chemical space of known drugs, if the desired molecules were not found, the process ends. In this work, we introduce a curiosity-driven method to force the model to navigate many parts of the chemical space, therefore, achieving higher desirability and diversity as well. At first, we train a recurrent neural network-based general molecular generator (G), then we fine-tune G to maximize curiosity and desirability. We define curiosity as the Tanimoto similarity between two generated molecules, a first molecule generated by G, and a second one generated by a copy of G (Gcopy). We only backpropagate the loss through G while keeping Gcopy unchanged. We benchmarked our approach against two desirable chemical properties related to drug-likeness and showed that the discovered chemical space can be significantly expanded, thus, discovering a higher number of desirable molecules with more diversity and potentially easier to synthesize. All Code and data used in this paper are available at https://github.com/amine179/Curiosity-RL-for-Drug-Design.