# Generating stable molecules using imitation and reinforcement learning

arXiv: https://arxiv.org/abs/2107.05007 · HTML: https://mlsanity.com/paper/2107.05007/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 9 of its 15 references that are in the map.
- First posted on arXiv 11 Jul 2021.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [MolGAN: An implicit generative model for small molecular graphs](https://mlsanity.com/paper/1805.11973/) (2018)
- [Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities](https://mlsanity.com/paper/2006.02425/) (2020)
- [Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules](https://mlsanity.com/paper/1906.00957/) (2019)
- [Multi-Objective De Novo Drug Design with Conditional Graph Generative Model](https://mlsanity.com/paper/1801.07299/) (2018)
- [Symmetry-Aware Actor-Critic for 3D Molecular Design](https://mlsanity.com/paper/2011.12747/) (2020)
- [Reinforcement Learning for Molecular Design Guided by Quantum Mechanics](https://mlsanity.com/paper/2002.07717/) (2020)
- [Permutation-Invariant Variational Autoencoder for Graph-Level Representation Learning](https://mlsanity.com/paper/2104.09856/) (2021)

## Built on by (most cited first, 9 of 9)

- [Inverse design of 3d molecular structures with conditional generative neural networks](https://mlsanity.com/paper/2109.04824/) (2021)
- [Top-N: Equivariant set and graph generation without exchangeability](https://mlsanity.com/paper/2110.02096/) (2021)
- [Graph Neural Networks in Modern AI-aided Drug Discovery](https://mlsanity.com/paper/2506.06915/) (2025)
- [Molecular relaxation by reverse diffusion with time step prediction](https://mlsanity.com/paper/2404.10935/) (2024)
- [Gradient-based grand canonical optimization enabled by graph neural networks with fractional atomic existence](https://mlsanity.com/paper/2507.19438/) (2025)
- [Reinforcement Learning for Chemical Ordering in Alloy Nanoparticles](https://mlsanity.com/paper/2511.12260/) (2025)
- [AtomComposer: Discovering Chemical Space from First Principles with Reinforcement Learning](https://mlsanity.com/paper/2605.28287/) (2026)
- [Cluster Generation via Deep Energy-Based Model](https://mlsanity.com/paper/2206.09002/) (2022)
- [Distributed Reinforcement Learning for Molecular Design: Antioxidant case](https://mlsanity.com/paper/2312.01267/) (2023)

## Abstract (arXiv)

Chemical space is routinely explored by machine learning methods to discover interesting molecules, before time-consuming experimental synthesizing is attempted. However, these methods often rely on a graph representation, ignoring 3D information necessary for determining the stability of the molecules. We propose a reinforcement learning approach for generating molecules in cartesian coordinates allowing for quantum chemical prediction of the stability. To improve sample-efficiency we learn basic chemical rules from imitation learning on the GDB-11 database to create an initial model applicable for all stoichiometries. We then deploy multiple copies of the model conditioned on a specific stoichiometry in a reinforcement learning setting. The models correctly identify low energy molecules in the database and produce novel isomers not found in the training set. Finally, we apply the model to larger molecules to show how reinforcement learning further refines the imitation learning model in domains far from the training data.