# URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment

arXiv: https://arxiv.org/abs/2607.04688 · HTML: https://mlsanity.com/paper/2607.04688/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 49 references that are in the map.
- First posted on arXiv 6 Jul 2026.

## Builds on (most cited first, 14 of 14)

- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [Learning to Plan Chemical Syntheses](https://mlsanity.com/paper/1708.04202/) (2017)
- [Retro*: Learning Retrosynthetic Planning with Neural Guided A* Search](https://mlsanity.com/paper/2006.15820/) (2020)
- [Re-evaluating Retrosynthesis Algorithms with Syntheseus](https://mlsanity.com/paper/2310.19796/) (2023)
- [RetroGraph: Retrosynthetic Planning with Graph Search](https://mlsanity.com/paper/2206.11477/) (2022)
- [Self-Improved Retrosynthetic Planning](https://mlsanity.com/paper/2106.04880/) (2021)
- [Chemical reasoning in LLMs unlocks strategy-aware synthesis planning and reaction mechanism elucidation](https://mlsanity.com/paper/2503.08537/) (2025)
- [Models Matter: The Impact of Single-Step Retrosynthesis on Synthesis Planning](https://mlsanity.com/paper/2308.05522/) (2023)
- [DirectMultiStep: Direct Route Generation for Multistep Retrosynthesis](https://mlsanity.com/paper/2405.13983/) (2024)
- [Synthelite: Chemist-aligned and feasibility-aware synthesis planning with LLMs](https://mlsanity.com/paper/2512.16424/) (2025)
- [Chemist-aligned retrosynthesis by ensembling diverse inductive bias models](https://mlsanity.com/paper/2412.05269/) (2024)
- [When Single Answer Is Not Enough: Rethinking Single-Step Retrosynthesis Benchmarks for LLMs](https://mlsanity.com/paper/2602.03554/) (2026)
- [Trustworthy Retrosynthesis: Eliminating Hallucinations with a Diverse Ensemble of Reaction Scorers](https://mlsanity.com/paper/2510.10645/) (2025)
- [Procrustean Bed for AI-Driven Retrosynthesis: A Unified Framework for Reproducible Evaluation](https://mlsanity.com/paper/2512.07079/) (2025)

## Built on by (most cited first, 2 of 2)

- [FREA: A Multi-Source Expert Benchmark for Reaction Feasibility Verification](https://mlsanity.com/paper/2610.06614/) (2026)
- [RetroPlanner: Multi-Route Expansion and Multi-Objective Reasoning for LLM-Based Retrosynthesis Planning](https://mlsanity.com/paper/2610.16391/) (2026)

## Abstract (arXiv)

Synthesis planning aiming to find pathways of reactions for a target molecule is one of the most important and challenging tasks in drug discovery. Recent progress has produced both specialized deep-learning retrosynthesis systems and general-purpose large language models, but objective comparison remains difficult due to the lack of flexible, chemically interpretable benchmarking protocols. In the current study, we are introducing the URSA (Utilitarian RetroSynthesis Assessment) evaluation framework that provides the opportunity to benchmark the synthetic routes not only from a formal perspective, such as convergence to commercially available starting materials, but also from a chemical plausibility perspective, mimicking the way expert chemists evaluate the reactions and routes. The study covers a comprehensive evaluation of both conventional end-to-end retrosynthesis solutions and LLMs for the synthesis planning task on a set of novel, diverse target molecules with undisclosed synthetic routes, which represent realistic tasks in the daily drug design routine. We find that while LLMs can support high-level strategic planning, they currently underperform specialized retrosynthesis models in reliably solving synthesis planning tasks.