# Entropy-Reinforced Planning with Large Language Models for Drug Discovery

arXiv: https://arxiv.org/abs/2406.07025 · HTML: https://mlsanity.com/paper/2406.07025/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 60 references that are in the map.
- First posted on arXiv 11 Jun 2024.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [ChatGPT for Robotics: Design Principles and Model Abilities](https://mlsanity.com/paper/2306.17582/) (2023)
- [Molecular De Novo Design through Deep Reinforcement Learning](https://mlsanity.com/paper/1704.07555/) (2017)
- [Optimization of Molecules via Deep Reinforcement Learning](https://mlsanity.com/paper/1810.08678/) (2018)
- [Byte Pair Encoding is Suboptimal for Language Model Pretraining](https://mlsanity.com/paper/2004.03720/) (2020)
- [Planning with Large Language Models for Code Generation](https://mlsanity.com/paper/2303.05510/) (2023)
- [Deep Reinforcement Learning for De-Novo Drug Design](https://mlsanity.com/paper/1711.10907/) (2017)
- [The Science of Detecting LLM-Generated Texts](https://mlsanity.com/paper/2303.07205/) (2023)
- [Reinforcement and Imitation Learning via Interactive No-Regret Learning](https://mlsanity.com/paper/1406.5979/) (2014)
- [Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models](https://mlsanity.com/paper/1705.10843/) (2017)
- [If beam search is the answer, what was the question?](https://mlsanity.com/paper/2010.02650/) (2020)
- [Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement Learning](https://mlsanity.com/paper/2004.12485/) (2020)
- [Machine Translation Decoding beyond Beam Search](https://mlsanity.com/paper/2104.05336/) (2021)
- [Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language](https://mlsanity.com/paper/2303.03363/) (2023)
- [PPL-MCTS: Constrained Textual Generation Through Discriminator-Guided MCTS Decoding](https://mlsanity.com/paper/2109.13582/) (2021)
- [ChemTS: An Efficient Python Library for de novo Molecular Generation](https://mlsanity.com/paper/1710.00616/) (2017)
- [ChatGPT-powered Conversational Drug Editing Using Retrieval and Domain Feedback](https://mlsanity.com/paper/2305.18090/) (2023)
- [Multi-task Language Modeling for Improving Speech Recognition of Rare Words](https://mlsanity.com/paper/2011.11715/) (2020)
- [Blending Imitation and Reinforcement Learning for Robust Policy Improvement](https://mlsanity.com/paper/2310.01737/) (2023)
- [C5T5: Controllable Generation of Organic Molecules with Transformers](https://mlsanity.com/paper/2108.10307/) (2021)
- [Retrosynthetic Planning with Experience-Guided Monte Carlo Tree Search](https://mlsanity.com/paper/2112.06028/) (2021)
- [To Beam Or Not To Beam: That is a Question of Cooperation for Language GANs](https://mlsanity.com/paper/2106.06363/) (2021)
- [Active Policy Improvement from Multiple Black-box Oracles](https://mlsanity.com/paper/2306.10259/) (2023)

## Built on by (most cited first, 6 of 6)

- [Decomposing the Entropy-Performance Exchange: The Missing Keys to Unlocking Effective Reinforcement Learning](https://mlsanity.com/paper/2508.02260/) (2025)
- [Active Advantage-Aligned Online Reinforcement Learning with Offline Data](https://mlsanity.com/paper/2502.07937/) (2025)
- [Binding Affinity Prediction: From Conventional to Machine Learning-Based Approaches](https://mlsanity.com/paper/2410.00709/) (2024)
- [Monte Carlo Tree Diffusion with Multiple Experts for Protein Design](https://mlsanity.com/paper/2509.15796/) (2025)
- [FedTDP: A Privacy-Preserving and Unified Framework for Trajectory Data Preparation via Federated Learning](https://mlsanity.com/paper/2505.05155/) (2025)
- [Leveraging generative hallucination and biophysics-informed modeling for unified biomolecular sequence-structure co-design](https://mlsanity.com/paper/2608.17381/) (2026)

## Abstract (arXiv)

The objective of drug discovery is to identify chemical compounds that possess specific pharmaceutical properties toward a binding target. Existing large language models (LLMS) can achieve high token matching scores in terms of likelihood for molecule generation. However, relying solely on LLM decoding often results in the generation of molecules that are either invalid due to a single misused token, or suboptimal due to unbalanced exploration and exploitation as a consequence of the LLMs prior experience. Here we propose ERP, Entropy-Reinforced Planning for Transformer Decoding, which employs an entropy-reinforced planning algorithm to enhance the Transformer decoding process and strike a balance between exploitation and exploration. ERP aims to achieve improvements in multiple properties compared to direct sampling from the Transformer. We evaluated ERP on the SARS-CoV-2 virus (3CLPro) and human cancer cell target protein (RTCB) benchmarks and demonstrated that, in both benchmarks, ERP consistently outperforms the current state-of-the-art algorithm by 1-5 percent, and baselines by 5-10 percent, respectively. Moreover, such improvement is robust across Transformer models trained with different objectives. Finally, to further illustrate the capabilities of ERP, we tested our algorithm on three code generation benchmarks and outperformed the current state-of-the-art approach as well. Our code is publicly available at: https://github.com/xuefeng-cs/ERP.