# RLSynC: Offline-Online Reinforcement Learning for Synthon Completion

arXiv: https://arxiv.org/abs/2309.02671 · HTML: https://mlsanity.com/paper/2309.02671/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 47 references that are in the map.
- First posted on arXiv 6 Sep 2023.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [AWAC: Accelerating Online Reinforcement Learning with Offline Datasets](https://mlsanity.com/paper/2006.09359/) (2020)
- [Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation](https://mlsanity.com/paper/1806.02473/) (2018)
- [Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Prediction](https://mlsanity.com/paper/1811.02633/) (2018)
- [Optimization of Molecules via Deep Reinforcement Learning](https://mlsanity.com/paper/1810.08678/) (2018)
- [Retrosynthesis Prediction with Conditional Graph Logic Network](https://mlsanity.com/paper/2001.01408/) (2020)
- [Policy Expansion for Bridging Offline-to-Online Reinforcement Learning](https://mlsanity.com/paper/2302.00935/) (2023)
- [A Graph to Graphs Framework for Retrosynthesis Prediction](https://mlsanity.com/paper/2003.12725/) (2020)
- [Retro*: Learning Retrosynthetic Planning with Neural Guided A* Search](https://mlsanity.com/paper/2006.15820/) (2020)
- [RetroXpert: Decompose Retrosynthesis Prediction like a Chemist](https://mlsanity.com/paper/2011.02893/) (2020)
- [Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction prediction](https://mlsanity.com/paper/2110.09681/) (2021)
- [Learning Graph Models for Retrosynthesis Prediction](https://mlsanity.com/paper/2006.07038/) (2020)
- [Root-aligned SMILES: A Tight Representation for Chemical Reaction Prediction](https://mlsanity.com/paper/2203.11444/) (2022)
- [MechRetro is a chemical-mechanism-driven graph learning framework for interpretable retrosynthesis prediction and pathway planning](https://mlsanity.com/paper/2210.02630/) (2022)
- [MARS: A Motif-based Autoregressive Model for Retrosynthesis Prediction](https://mlsanity.com/paper/2209.13178/) (2022)
- [RCsearcher: Reaction Center Identification in Retrosynthesis via Deep Q-Learning](https://mlsanity.com/paper/2301.12071/) (2023)

## Built on by (most cited first, 4 of 4)

- [LARC: Towards Human-level Constrained Retrosynthesis Planning through an Agentic Framework](https://mlsanity.com/paper/2508.11860/) (2025)
- [Preference Optimization for Molecule Synthesis with Conditional Residual Energy-based Models](https://mlsanity.com/paper/2406.02066/) (2024)
- [SynCoGen: Synthesizable 3D Molecule Generation via Joint Reaction and Coordinate Modeling](https://mlsanity.com/paper/2507.11818/) (2025)
- [Single-step Retrosynthesis via Reinforcement Learning from Forward Feedback](https://mlsanity.com/paper/2610.15141/) (2026)

## Abstract (arXiv)

Retrosynthesis is the process of determining the set of reactant molecules that can react to form a desired product. Semi-template-based retrosynthesis methods, which imitate the reverse logic of synthesis reactions, first predict the reaction centers in the products, and then complete the resulting synthons back into reactants. We develop a new offline-online reinforcement learning method RLSynC for synthon completion in semi-template-based methods. RLSynC assigns one agent to each synthon, all of which complete the synthons by conducting actions step by step in a synchronized fashion. RLSynC learns the policy from both offline training episodes and online interactions, which allows RLSynC to explore new reaction spaces. RLSynC uses a standalone forward synthesis model to evaluate the likelihood of the predicted reactants in synthesizing a product, and thus guides the action search. Our results demonstrate that RLSynC can outperform state-of-the-art synthon completion methods with improvements as high as 14.9%, highlighting its potential in synthesis planning.