# REINFORCE-ING Chemical Language Models for Drug Discovery

arXiv: https://arxiv.org/abs/2501.15971 · HTML: https://mlsanity.com/paper/2501.15971/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 51 references that are in the map.
- First posted on arXiv 27 Jan 2025.

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [GuacaMol: Benchmarking Models for De Novo Molecular Design](https://mlsanity.com/paper/1811.09621/) (2018)
- [Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models](https://mlsanity.com/paper/1811.12823/) (2018)
- [Molecular De Novo Design through Deep Reinforcement Learning](https://mlsanity.com/paper/1704.07555/) (2017)
- [Deep Reinforcement Learning for De-Novo Drug Design](https://mlsanity.com/paper/1711.10907/) (2017)
- [SynFlowNet: Design of Diverse and Novel Molecules with Synthesis Constraints](https://mlsanity.com/paper/2405.01155/) (2024)
- [Guided Exploration with Proximal Policy Optimization using a Single Demonstration](https://mlsanity.com/paper/2007.03328/) (2020)
- [ACEGEN: Reinforcement learning of generative chemical agents for drug discovery](https://mlsanity.com/paper/2405.04657/) (2024)
- [Re-evaluating sample efficiency in de novo molecule generation](https://mlsanity.com/paper/2212.01385/) (2022)
- [Diversity-Aware Reinforcement Learning for de novo Drug Design](https://mlsanity.com/paper/2410.10431/) (2024)

## Built on by (most cited first, 3 of 3)

- [Test-Time Training Scaling Laws for Chemical Exploration in Drug Design](https://mlsanity.com/paper/2501.19153/) (2025)
- [TSSR: Two-Stage Swap-Reward-Driven Reinforcement Learning for Character-Level SMILES Generation](https://mlsanity.com/paper/2601.04521/) (2026)
- [Elite-Weighted Supervised Fine-tuning for Goal-Directed Molecular Optimization](https://mlsanity.com/paper/2609.00189/) (2026)

## Abstract (arXiv)

Chemical language models, combined with reinforcement learning (RL), have shown significant promise to efficiently traverse large chemical spaces for drug discovery. However, the performance of various RL algorithms and their best practices for practical drug discovery are still unclear. Here, starting from the principles of the REINFORCE algorithm, we investigate the effect of different components from RL theory including experience replay, hill-climbing, baselines to reduce variance, and alternative reward shaping. We propose a new regularization method more aligned to REINFORCE than current standard practices, and demonstrate how RL hyperparameters can be fine-tuned for effectiveness and efficiency. Lastly, we apply our learnings to practical drug discovery by demonstrating enhanced learning efficiency on frontier binding affinity models by using Boltz2 as a reward model. We share our RL models used in the ACEGEN repository, and hope the experiments here act as a guide to researchers applying RL to chemical language models for drug discovery.