# Scalable Fragment-Based 3D Molecular Design with Reinforcement Learning

arXiv: https://arxiv.org/abs/2202.00658 · HTML: https://mlsanity.com/paper/2202.00658/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 35 references that are in the map.
- First posted on arXiv 1 Feb 2022.

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [GemNet: Universal Directional Graph Neural Networks for Molecules](https://mlsanity.com/paper/2106.08903/) (2021)
- [Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models](https://mlsanity.com/paper/1705.10843/) (2017)
- [GeoMol: Torsional Geometric Generation of Molecular 3D Conformer Ensembles](https://mlsanity.com/paper/2106.07802/) (2021)
- [Learning Multimodal Graph-to-Graph Translation for Molecular Optimization](https://mlsanity.com/paper/1812.01070/) (2018)
- [Symmetry-Aware Actor-Critic for 3D Molecular Design](https://mlsanity.com/paper/2011.12747/) (2020)
- [Neural Message Passing on High Order Paths](https://mlsanity.com/paper/2002.10413/) (2020)
- [Discrete Object Generation with Reversible Inductive Construction](https://mlsanity.com/paper/1907.08268/) (2019)
- [TorsionNet: A Reinforcement Learning Approach to Sequential Conformer Search](https://mlsanity.com/paper/2006.07078/) (2020)
- [Message Passing Networks for Molecules with Tetrahedral Chirality](https://mlsanity.com/paper/2012.00094/) (2020)
- [Generating Focussed Molecule Libraries for Drug Discovery with Recurrent Neural Networks](https://mlsanity.com/paper/1701.01329/) (2017)
- [Fragment-based Sequential Translation for Molecular Optimization](https://mlsanity.com/paper/2111.01009/) (2021)

## Built on by (most cited first, 11 of 11)

- [Group SELFIES: A Robust Fragment-Based Molecular String Representation](https://mlsanity.com/paper/2211.13322/) (2022)
- [DiffMS: Diffusion Generation of Molecules Conditioned on Mass Spectra](https://mlsanity.com/paper/2502.09571/) (2025)
- [Graph Neural Networks for Molecules](https://mlsanity.com/paper/2209.05582/) (2022)
- [Equivariant Shape-Conditioned Generation of 3D Molecules for Ligand-Based Drug Design](https://mlsanity.com/paper/2210.04893/) (2022)
- [Atom-by-atom protein generation and beyond with language models](https://mlsanity.com/paper/2308.09482/) (2023)
- [AUTODIFF: Autoregressive Diffusion Modeling for Structure-based Drug Design](https://mlsanity.com/paper/2404.02003/) (2024)
- [Machine Learning for Screening Large Organic Molecules](https://mlsanity.com/paper/2211.15415/) (2022)
- [Active Causal Learning for Decoding Chemical Complexities with Targeted Interventions](https://mlsanity.com/paper/2404.04224/) (2024)
- [A large language model-type architecture for high-dimensional molecular potential energy surfaces](https://mlsanity.com/paper/2412.03831/) (2024)
- [Reinforcement Learning for Chemical Ordering in Alloy Nanoparticles](https://mlsanity.com/paper/2511.12260/) (2025)
- [AtomComposer: Discovering Chemical Space from First Principles with Reinforcement Learning](https://mlsanity.com/paper/2605.28287/) (2026)

## Abstract (arXiv)

Machine learning has the potential to automate molecular design and drastically accelerate the discovery of new functional compounds. Towards this goal, generative models and reinforcement learning (RL) using string and graph representations have been successfully used to search for novel molecules. However, these approaches are limited since their representations ignore the three-dimensional (3D) structure of molecules. In fact, geometry plays an important role in many applications in inverse molecular design, especially in drug discovery. Thus, it is important to build models that can generate molecular structures in 3D space based on property-oriented geometric constraints. To address this, one approach is to generate molecules as 3D point clouds by sequentially placing atoms at locations in space -- this allows the process to be guided by physical quantities such as energy or other properties. However, this approach is inefficient as placing individual atoms makes the exploration unnecessarily deep, limiting the complexity of molecules that can be generated. Moreover, when optimizing a molecule, organic and medicinal chemists use known fragments and functional groups, not single atoms. We introduce a novel RL framework for scalable 3D design that uses a hierarchical agent to build molecules by placing molecular substructures sequentially in 3D space, thus attempting to build on the existing human knowledge in the field of molecular design. In a variety of experiments with different substructures, we show that our agent, guided only by energy considerations, can efficiently learn to produce molecules with over 100 atoms from many distributions including drug-like molecules, organic LED molecules, and biomolecules.