# Procrustean Bed for AI-Driven Retrosynthesis: A Unified Framework for Reproducible Evaluation

arXiv: https://arxiv.org/abs/2512.07079 · HTML: https://mlsanity.com/paper/2512.07079/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 8 Dec 2025.

## Builds on (most cited first, 16 of 16)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Learning to Plan Chemical Syntheses](https://mlsanity.com/paper/1708.04202/) (2017)
- [Retrosynthesis Prediction with Conditional Graph Logic Network](https://mlsanity.com/paper/2001.01408/) (2020)
- [Predicting Organic Reaction Outcomes with Weisfeiler-Lehman Network](https://mlsanity.com/paper/1709.04555/) (2017)
- [Retro*: Learning Retrosynthetic Planning with Neural Guided A* Search](https://mlsanity.com/paper/2006.15820/) (2020)
- [Re-evaluating Retrosynthesis Algorithms with Syntheseus](https://mlsanity.com/paper/2310.19796/) (2023)
- [RetroGraph: Retrosynthetic Planning with Graph Search](https://mlsanity.com/paper/2206.11477/) (2022)
- [Double-Ended Synthesis Planning with Goal-Constrained Bidirectional Search](https://mlsanity.com/paper/2407.06334/) (2024)
- [Self-Improved Retrosynthetic Planning](https://mlsanity.com/paper/2106.04880/) (2021)
- [Retrosynthetic Planning with Experience-Guided Monte Carlo Tree Search](https://mlsanity.com/paper/2112.06028/) (2021)
- [FusionRetro: Molecule Representation Fusion via In-Context Learning for Retrosynthetic Planning](https://mlsanity.com/paper/2209.15315/) (2022)
- [RetroGFN: Diverse and Feasible Retrosynthesis using GFlowNets](https://mlsanity.com/paper/2406.18739/) (2024)
- [DirectMultiStep: Direct Route Generation for Multistep Retrosynthesis](https://mlsanity.com/paper/2405.13983/) (2024)
- [SynLlama: Generating Synthesizable Molecules and Their Analogs with Large Language Models](https://mlsanity.com/paper/2503.12602/) (2025)
- [Chemist-aligned retrosynthesis by ensembling diverse inductive bias models](https://mlsanity.com/paper/2412.05269/) (2024)

## Built on by (most cited first, 4 of 4)

- [When Single Answer Is Not Enough: Rethinking Single-Step Retrosynthesis Benchmarks for LLMs](https://mlsanity.com/paper/2602.03554/) (2026)
- [URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment](https://mlsanity.com/paper/2607.04688/) (2026)
- [Project Ariadne: Prompt-Conditioned Route Generation for Synthesis Planning](https://mlsanity.com/paper/2606.24184/) (2026)
- [Strategy-first synthesis planning for complex natural products](https://mlsanity.com/paper/2608.07454/) (2026)

## Abstract (arXiv)

Progress in computer-aided synthesis planning (CASP) is obscured by the lack of standardized evaluation infrastructure and the reliance on metrics that prioritize topological completion over chemical validity. We introduce RetroCast, a unified evaluation suite that standardizes heterogeneous model outputs into a common schema to enable statistically rigorous, apples-to-apples comparison. The framework includes a reproducible benchmarking pipeline with stratified sampling and bootstrapped confidence intervals, accompanied by SynthArena, an interactive platform for qualitative route inspection. We utilize this infrastructure to evaluate leading search-based and sequence-based algorithms on a new suite of standardized benchmarks. Our analysis reveals a divergence between "solvability" (stock-termination rate) and route quality; high solvability scores often mask chemical invalidity or fail to correlate with the reproduction of experimental ground truths. Furthermore, we identify a "complexity cliff" in which search-based methods, despite high solvability rates, exhibit a sharp performance decay in reconstructing long-range synthetic plans compared to sequence-based approaches. We release the full framework, benchmark definitions, and a standardized database of model predictions to support transparent and reproducible development in the field.