# Grammars and reinforcement learning for molecule optimization

arXiv: https://arxiv.org/abs/1811.11222 · HTML: https://mlsanity.com/paper/1811.11222/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2018.

## Built on by (most cited first, 3 of 3)

- [Automatic design of novel potential 3CLpro and PLpro inhibitors](https://mlsanity.com/paper/2101.11890/) (2021)
- [Recursive Reinforcement Learning](https://mlsanity.com/paper/2206.11430/) (2022)
- [Constrained Goal-directed Planar Graph Generation with Grammar-based Reinforcement Learning](https://mlsanity.com/paper/2610.06244/) (2026)

## Abstract (arXiv)

We seek to automate the design of molecules based on specific chemical properties. Our primary contributions are a simpler method for generating SMILES strings guaranteed to be chemically valid, using a combination of a new context-free grammar for SMILES and additional masking logic; and casting the molecular property optimization as a reinforcement learning problem, specifically best-of-batch policy gradient applied to a Transformer model architecture. This approach uses substantially fewer model steps per atom than earlier approaches, thus enabling generation of larger molecules, and beats previous state-of-the art baselines by a significant margin. Applying reinforcement learning to a combination of a custom context-free grammar with additional masking to enforce non-local constraints is applicable to any optimization of a graph structure under a mixture of local and nonlocal constraints.