# Retrosynthesis Planning via Worst-path Policy Optimisation in Tree-structured MDPs

arXiv: https://arxiv.org/abs/2509.10504 · HTML: https://mlsanity.com/paper/2509.10504/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 1 Sep 2025.

## Builds on (most cited first, 21 of 21)

- [Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning](https://mlsanity.com/paper/1910.00177/) (2019)
- [AWAC: Accelerating Online Reinforcement Learning with Offline Datasets](https://mlsanity.com/paper/2006.09359/) (2020)
- [Analyzing Learned Molecular Representations for Property Prediction](https://mlsanity.com/paper/1904.01561/) (2019)
- [Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learning](https://mlsanity.com/paper/2002.08396/) (2020)
- [Learning to Plan Chemical Syntheses](https://mlsanity.com/paper/1708.04202/) (2017)
- [Self-Imitation Learning](https://mlsanity.com/paper/1806.05635/) (2018)
- [Online and Offline Reinforcement Learning by Planning with a Learned Model](https://mlsanity.com/paper/2104.06294/) (2021)
- [Molecule Edit Graph Attention Network: Modeling Chemical Reactions as Sequences of Graph Edits](https://mlsanity.com/paper/2006.15426/) (2020)
- [Retro*: Learning Retrosynthetic Planning with Neural Guided A* Search](https://mlsanity.com/paper/2006.15820/) (2020)
- [RetroXpert: Decompose Retrosynthesis Prediction like a Chemist](https://mlsanity.com/paper/2011.02893/) (2020)
- [Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction prediction](https://mlsanity.com/paper/2110.09681/) (2021)
- [Learning Graph Models for Retrosynthesis Prediction](https://mlsanity.com/paper/2006.07038/) (2020)
- [Re-evaluating Retrosynthesis Algorithms with Syntheseus](https://mlsanity.com/paper/2310.19796/) (2023)
- [RetroGraph: Retrosynthetic Planning with Graph Search](https://mlsanity.com/paper/2206.11477/) (2022)
- [Self-Improved Retrosynthetic Planning](https://mlsanity.com/paper/2106.04880/) (2021)
- [Retrosynthetic Planning with Dual Value Networks](https://mlsanity.com/paper/2301.13755/) (2023)
- [GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Models](https://mlsanity.com/paper/2310.20025/) (2023)
- [Supported Trust Region Optimization for Offline Reinforcement Learning](https://mlsanity.com/paper/2311.08935/) (2023)
- [Retrosynthetic Planning with Experience-Guided Monte Carlo Tree Search](https://mlsanity.com/paper/2112.06028/) (2021)
- [MechRetro is a chemical-mechanism-driven graph learning framework for interpretable retrosynthesis prediction and pathway planning](https://mlsanity.com/paper/2210.02630/) (2022)
- [Learning on One Mode: Addressing Multi-modality in Offline Reinforcement Learning](https://mlsanity.com/paper/2412.03258/) (2024)

## Built on by (most cited first, 2 of 2)

- [From Feasible to Practical: Pareto-Optimal Synthesis Planning](https://mlsanity.com/paper/2605.07521/) (2026)
- [Project Ariadne: Prompt-Conditioned Route Generation for Synthesis Planning](https://mlsanity.com/paper/2606.24184/) (2026)

## Abstract (arXiv)

Retrosynthesis planning aims to decompose target molecules into available building blocks, forming a synthetic tree where each internal node represents an intermediate compound and each leaf ideally corresponds to a purchasable reactant. However, this tree becomes invalid if any leaf node is not a valid building block, making the planning process vulnerable to the "weakest link" in the synthetic route. Existing methods often optimise for average performance across branches, failing to account for this worst-case sensitivity. In this paper, we reframe retrosynthesis as a worst-path optimisation problem within tree-structured Markov Decision Processes (MDPs). We prove that this formulation admits a unique optimal solution and provides monotonic improvement guarantees. Building on this insight, we introduce Interactive Retrosynthesis Planning (InterRetro), a method that interacts with the tree MDP, learns a value function for worst-path outcomes, and improves its policy through self-imitation, preferentially reinforcing past decisions with high estimated advantage. Empirically, InterRetro achieves state-of-the-art results - solving 100% of targets on the Retro*-190 benchmark, shortening synthetic routes by 4.9%, and achieving promising performance using only 10% of the training data.