# When Single Answer Is Not Enough: Rethinking Single-Step Retrosynthesis Benchmarks for LLMs

arXiv: https://arxiv.org/abs/2602.03554 · HTML: https://mlsanity.com/paper/2602.03554/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 15 of its 91 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)

## Builds on (most cited first, 15 of 15)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [Kimi K2: Open Agentic Intelligence](https://mlsanity.com/paper/2507.20534/) (2025)
- [ChemLLM: A Chemical Large Language Model](https://mlsanity.com/paper/2402.06852/) (2024)
- [Evaluating Large Language Models in Scientific Discovery](https://mlsanity.com/paper/2512.15567/) (2025)
- [LFM2 Technical Report](https://mlsanity.com/paper/2511.23404/) (2025)
- [Nature Language Model: Deciphering the Language of Nature for Scientific Discovery](https://mlsanity.com/paper/2502.07527/) (2025)
- [Retrosynthesis Prediction with Local Template Retrieval](https://mlsanity.com/paper/2306.04123/) (2023)
- [Synthelite: Chemist-aligned and feasibility-aware synthesis planning with LLMs](https://mlsanity.com/paper/2512.16424/) (2025)
- [SUPERChem: A Multimodal Reasoning Benchmark in Chemistry](https://mlsanity.com/paper/2512.01274/) (2025)
- [Trustworthy Retrosynthesis: Eliminating Hallucinations with a Diverse Ensemble of Reaction Scorers](https://mlsanity.com/paper/2510.10645/) (2025)
- [Atom-anchored LLMs speak Chemistry: A Retrosynthesis Demonstration](https://mlsanity.com/paper/2510.16590/) (2025)
- [Procrustean Bed for AI-Driven Retrosynthesis: A Unified Framework for Reproducible Evaluation](https://mlsanity.com/paper/2512.07079/) (2025)
- [RetroDFM-R: Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learning](https://mlsanity.com/paper/2507.17448/) (2025)

## Built on by (most cited first, 5 of 5)

- [URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment](https://mlsanity.com/paper/2607.04688/) (2026)
- [Learning Mechanistic Reasoning for Chemical Reactions with Large Language Models](https://mlsanity.com/paper/2607.12771/) (2026)
- [MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery](https://mlsanity.com/paper/2603.03517/) (2026)
- [Coverage Before Control: Route-Instruction Grounding and Steering for Controllable Retrosynthesis](https://mlsanity.com/paper/2609.39955/) (2026)
- [FREA: A Multi-Source Expert Benchmark for Reaction Feasibility Verification](https://mlsanity.com/paper/2610.06614/) (2026)

## Abstract (arXiv)

Recent progress has expanded the use of large language models (LLMs) in drug discovery, including synthesis planning. However, objective evaluation of retrosynthesis performance remains limited. Existing benchmarks and metrics typically rely on published synthetic procedures and Top-K accuracy based on single ground-truth, which does not capture the open-ended nature of real-world synthesis planning. We propose a new benchmarking framework for single-step retrosynthesis that evaluates both general-purpose and chemistry-specialized LLMs using ChemCensor, a novel metric for chemical plausibility. By emphasizing plausibility over exact match, this approach better aligns with human synthesis planning practices. We also introduce CREED, a novel dataset comprising millions of ChemCensor-validated reaction records for LLM training, and use it to train a model that improves over the LLM baselines under this benchmark.