# Molecular De Novo Design through Transformer-based Reinforcement Learning

arXiv: https://arxiv.org/abs/2310.05365 · HTML: https://mlsanity.com/paper/2310.05365/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 59 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Builds on (most cited first, 19 of 19)

- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [GuacaMol: Benchmarking Models for De Novo Molecular Design](https://mlsanity.com/paper/1811.09621/) (2018)
- [Molecular De Novo Design through Deep Reinforcement Learning](https://mlsanity.com/paper/1704.07555/) (2017)
- [Optimization of Molecules via Deep Reinforcement Learning](https://mlsanity.com/paper/1810.08678/) (2018)
- [Sample Efficiency Matters: A Benchmark for Practical Molecular Optimization](https://mlsanity.com/paper/2206.12411/) (2022)
- [Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems](https://mlsanity.com/paper/2307.08423/) (2023)
- [ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendations](https://mlsanity.com/paper/1908.01425/) (2019)
- [Accelerating high-throughput virtual screening through molecular pool-based active learning](https://mlsanity.com/paper/2012.07127/) (2020)
- [BOSS: Bayesian Optimization over String Spaces](https://mlsanity.com/paper/2010.00979/) (2020)
- [Differentiable Scaffolding Tree for Molecular Optimization](https://mlsanity.com/paper/2109.10469/) (2021)
- [MIMOSA: Multi-constraint Molecule Sampling for Molecule Optimization](https://mlsanity.com/paper/2010.02318/) (2020)
- [Reinforced Genetic Algorithm for Structure-based Drug Design](https://mlsanity.com/paper/2211.16508/) (2022)
- [Amortized Tree Generation for Bottom-up Synthesis Planning and Synthesizable Molecular Design](https://mlsanity.com/paper/2110.06389/) (2021)
- [Barking up the right tree: an approach to search over molecule synthesis DAGs](https://mlsanity.com/paper/2012.11522/) (2020)
- [Deep Molecular Dreaming: Inverse machine learning for de-novo molecular design and interpretability with surjective representations](https://mlsanity.com/paper/2012.09712/) (2020)
- [Generating Focussed Molecule Libraries for Drug Discovery with Recurrent Neural Networks](https://mlsanity.com/paper/1701.01329/) (2017)

## Built on by (most cited first, 3 of 3)

- [Saturn: Sample-efficient Generative Molecular Design using Memory Manipulation](https://mlsanity.com/paper/2405.17066/) (2024)
- [Sample Efficient Generative Optimization for Molecular Design](https://mlsanity.com/paper/2607.12488/) (2026)
- [MolRGen: A Training and Evaluation Setting for De Novo Molecular Generation with Reasonning Models](https://mlsanity.com/paper/2603.18256/) (2026)

## Abstract (arXiv)

In this work, we introduce a method to fine-tune a Transformer-based generative model for molecular de novo design. Leveraging the superior sequence learning capacity of Transformers over Recurrent Neural Networks (RNNs), our model can generate molecular structures with desired properties effectively. In contrast to the traditional RNN-based models, our proposed method exhibits superior performance in generating compounds predicted to be active against various biological targets, capturing long-term dependencies in the molecular structure sequence. The model's efficacy is demonstrated across numerous tasks, including generating analogues to a query structure and producing compounds with particular attributes, outperforming the baseline RNN-based methods. Our approach can be used for scaffold hopping, library expansion starting from a single molecule, and generating compounds with high predicted activity against biological targets.