# Curiosity in exploring chemical space: Intrinsic rewards for deep molecular reinforcement learning

arXiv: https://arxiv.org/abs/2012.11293 · HTML: https://mlsanity.com/paper/2012.11293/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2020.

## Built on by (most cited first, 16 of 16)

- [SELFIES and the future of molecular string representations](https://mlsanity.com/paper/2204.00056/) (2022)
- [On scientific understanding with artificial intelligence](https://mlsanity.com/paper/2204.01467/) (2022)
- [Hit and Lead Discovery with Explorative RL and Fragment-based Molecule Generation](https://mlsanity.com/paper/2110.01219/) (2021)
- [DOCKSTRING: easy molecular docking yields better benchmarks for ligand design](https://mlsanity.com/paper/2110.15486/) (2021)
- [Global Reinforcement Learning: Beyond Linear and Convex Rewards via Submodular Semi-gradient Methods](https://mlsanity.com/paper/2407.09905/) (2024)
- [Assigning Confidence to Molecular Property Prediction](https://mlsanity.com/paper/2102.11439/) (2021)
- [Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges](https://mlsanity.com/paper/2308.00031/) (2023)
- [Recent advances in the Self-Referencing Embedding Strings (SELFIES) library](https://mlsanity.com/paper/2302.03620/) (2023)
- [Mol-AIR: Molecular Reinforcement Learning with Adaptive Intrinsic Rewards for Goal-directed Molecular Generation](https://mlsanity.com/paper/2403.20109/) (2024)
- [Geometric Active Exploration in Markov Decision Processes: the Benefit of Abstraction](https://mlsanity.com/paper/2407.13364/) (2024)
- [Curiosity Driven Exploration to Optimize Structure-Property Learning in Microscopy](https://mlsanity.com/paper/2504.20011/) (2025)
- [Automated Gadget Discovery in Science](https://mlsanity.com/paper/2212.12743/) (2022)
- [Test-Time Training Scaling Laws for Chemical Exploration in Drug Design](https://mlsanity.com/paper/2501.19153/) (2025)
- [Sample Efficient Reinforcement Learning by Automatically Learning to Compose Subtasks](https://mlsanity.com/paper/2401.14226/) (2024)
- [Rethinking Molecular Design: Integrating Latent Variable and Auto-Regressive Models for Goal Directed Generation](https://mlsanity.com/paper/2409.00046/) (2024)
- [Let CSP Be Your ANCHOR: Adaptive Crystal Search over Frozen Structure Priors](https://mlsanity.com/paper/2609.33407/) (2026)

## Abstract (arXiv)

Computer-aided design of molecules has the potential to disrupt the field of drug and material discovery. Machine learning, and deep learning, in particular, have been topics where the field has been developing at a rapid pace. Reinforcement learning is a particularly promising approach since it allows for molecular design without prior knowledge. However, the search space is vast and efficient exploration is desirable when using reinforcement learning agents. In this study, we propose an algorithm to aid efficient exploration. The algorithm is inspired by a concept known in the literature as curiosity. We show on three benchmarks that a curious agent finds better performing molecules. This indicates an exciting new research direction for reinforcement learning agents that can explore the chemical space out of their own motivation. This has the potential to eventually lead to unexpected new molecules that no human has thought about so far.