mlsanityData to 14 Oct 2026

Searching for High-Value Molecules Using Reinforcement Learning and Transformers

arXiv 2310.02902 ↗First posted 4 Oct 2023cs.LG cond-mat.mtrl-sci cs.AIcited by 18

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) over text representations can be effective for finding high-value policies that can search over graphs. However, RL requires careful structuring of the search space and algorithm design to be effective in this challenge. Through extensive experiments, we explore how different design choices for text grammar and algorithmic choices for training can affect an RL policy's ability to generate molecules with desired properties. We arrive at a new RL-based molecular design algorithm (ChemRLformer) and perform a thorough analysis using 25 molecule design tasks, including computationally complex protein docking simulations. From this analysis, we discover unique insights in this problem space and show that ChemRLformer achieves state-of-the-art performance while being more straightforward than prior work by demystifying which design choices are actually helpful for text-based molecule design.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 53 of its 82 references that are in the map. First posted on arXiv 4 Oct 2023.

Builds on

53 of 82 references.

References · 53

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  7. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  8. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  9. 2016Concrete Problems in AI Safetycited by 1,642
  10. 2016Generative Adversarial Imitation Learningcited by 1,597
  11. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
  12. 2022Galactica: A Large Language Model for Sciencecited by 744
38 more
  1. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 652
  2. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 392
  3. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  4. 2018Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generationcited by 364
  5. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 297
  6. 2019Stabilizing Transformers for Reinforcement Learningcited by 270
  7. 2017Molecular De Novo Design through Deep Reinforcement Learningcited by 267
  8. 2021Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Developmentcited by 264
  9. 2022Defining and Characterizing Reward Hackingcited by 203
  10. 2018Optimization of Molecules via Deep Reinforcement Learningcited by 200
  11. 2022Translation between Molecules and Natural Languagecited by 177
  12. 2022Sample Efficiency Matters: A Benchmark for Practical Molecular Optimizationcited by 155
  13. 2021Robustness of Graph Neural Networks at Scalecited by 100
  14. 2022Exploring Chemical Space with Score-based Out-of-distribution Generationcited by 95
  15. 2020Molecular representation learning with language models and domain-relevant auxiliary taskscited by 94
  16. 2022SELFIES and the future of molecular string representationscited by 76
  17. 2022Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizescited by 73
  18. 2019Recurrent Neural Networks (RNNs): A gentle Introduction and Overviewcited by 68
  19. 2021Hit and Lead Discovery with Explorative RL and Fragment-based Molecule Generationcited by 68
  20. 2021Keeping it Simple: Language Models can learn Complex Molecular Distributionscited by 68
  21. 2018Reinforcement Learning for Improving Agent Designcited by 56
  22. 2018Jointly Learning to Construct and Control Agents using Deep Reinforcement Learningcited by 55
  23. 2020BOSS: Bayesian Optimization over String Spacescited by 54
  24. 2018Policy Transfer with Strategy Optimizationcited by 43
  25. 2023Proto-Value Networks: Scaling Representation Learning with Auxiliary Taskscited by 42
  26. 2019Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learningcited by 37
  27. 2017Molecular Generation with Recurrent Neural Networks (RNNs)cited by 36
  28. 2021Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Studycited by 36
  29. 2019Deep Representation Learning for Social Network Analysiscited by 35
  30. 2021DOCKSTRING: easy molecular docking yields better benchmarks for ligand designcited by 35
  31. 2022Group SELFIES: A Robust Fragment-Based Molecular String Representationcited by 33
  32. 2020Hardware as Policy: Mechanical and Computational Co-Optimization using Deep Reinforcement Learningcited by 32
  33. 2021JANUS: Parallel Tempered Genetic Algorithm Guided by Deep Neural Networks for Inverse Molecular Designcited by 32
  34. 2021Data-Driven Offline Optimization For Architecting Hardware Acceleratorscited by 29
  35. 2017ChemGAN challenge for drug discovery: can AI reproduce natural chemical diversity?cited by 21
  36. 2021A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directionscited by 21
  37. 2020Deep Molecular Dreaming: Inverse machine learning for de-novo molecular design and interpretability with surjective representationscited by 20
  38. 2019All SMILES Variational Autoencodercited by 15

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 8; 2026: 3.

18 papers.

Sort:

By area:

  1. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 37
  2. 2024Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RLcited by 21
  3. 2024Less can be more for predicting properties with large language modelscited by 11
  4. 2024Can LLMs Generate Diverse Molecules? Towards Alignment with Structural Diversitycited by 9
  5. 2023AGaLiTe: Approximate Gated Linear Transformers for Online Reinforcement Learningcited by 6
  6. 2025Regulatory DNA sequence Design with Reinforcement Learningcited by 6
  7. 2025CrystalGym: A New Benchmark for Materials Discovery Using Reinforcement Learningcited by 4
  8. 2024Any-Property-Conditional Molecule Generation with Self-Criticism using Spanning Treescited by 3
  9. 2025Mol-MoE: Training Preference-Guided Routers for Molecule Generationcited by 3
  10. 2025Leveraging Partial SMILES Validation Scheme for Enhanced Drug Design in Reinforcement Learning Frameworkscited by 3
  11. 2024NPGPT: Natural Product-Like Compound Generation with GPT-based Chemical Language Modelscited by 2
  12. 2025A Multi-Fidelity Control Variate Approach for Policy Gradient Estimationcited by 2
6 more
  1. 2024FREED++: Improving RL Agents for Fragment-Based Molecule Generation by Thorough Reproductioncited by 1
  2. 2025TransDiffSBDD: Causality-Aware Multi-Modal Structure-Based Drug Designcited by 0
  3. 2025Dense and Diverse Goal Coverage in Multi Goal Reinforcement Learningcited by 0
  4. 2026How well can off-the-shelf LLMs elucidate molecular structures from mass spectra using chain-of-thought reasoning?cited by 0
  5. 2026MolLIBRA: Genetic Molecular Optimization with Multi-Fingerprint Surrogates and Text-Molecule Aligned Criticcited by 0
  6. 2026C-MORAL: Controllable Multi-Objective Molecular Optimization with Reinforcement Alignment for LLMscited by 0