# ConRetroBert: EMA Stabilized Dual Encoders for Template-Based Single-Step Retrosynthesis

arXiv: https://arxiv.org/abs/2605.12736 · HTML: https://mlsanity.com/paper/2605.12736/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 18 of 18)

- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering](https://mlsanity.com/paper/2106.05346/) (2021)
- [Retrosynthesis Prediction with Conditional Graph Logic Network](https://mlsanity.com/paper/2001.01408/) (2020)
- [Molecule Edit Graph Attention Network: Modeling Chemical Reactions as Sequences of Graph Edits](https://mlsanity.com/paper/2006.15426/) (2020)
- [Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction prediction](https://mlsanity.com/paper/2110.09681/) (2021)
- [Learning Graph Models for Retrosynthesis Prediction](https://mlsanity.com/paper/2006.07038/) (2020)
- [Re-evaluating Retrosynthesis Algorithms with Syntheseus](https://mlsanity.com/paper/2310.19796/) (2023)
- [Retrosynthesis Prediction with Local Template Retrieval](https://mlsanity.com/paper/2306.04123/) (2023)
- [Models Matter: The Impact of Single-Step Retrosynthesis on Synthesis Planning](https://mlsanity.com/paper/2308.05522/) (2023)
- [Modern Hopfield Networks for Few- and Zero-Shot Reaction Template Prediction](https://mlsanity.com/paper/2104.03279/) (2021)
- [RetroGFN: Diverse and Feasible Retrosynthesis using GFlowNets](https://mlsanity.com/paper/2406.18739/) (2024)
- [MechRetro is a chemical-mechanism-driven graph learning framework for interpretable retrosynthesis prediction and pathway planning](https://mlsanity.com/paper/2210.02630/) (2022)
- [RETRO SYNFLOW: Discrete Flow Matching for Accurate and Diverse Single-Step Retrosynthesis](https://mlsanity.com/paper/2506.04439/) (2025)

## Abstract (arXiv)

Template based single step retrosynthesis predicts reactants by selecting and applying an explicit reaction template, making each prediction traceable to a chemical transformation rule. This is useful for synthesis planning, but template based methods are often viewed as less competitive than template free models because template prediction is commonly formulated as global classification over a long tailed rule library. We argue that this weakness is not inherent to templates, but to the learning formulation. We present ConRetroBert, a dual encoder framework that reframes template based retrosynthesis as dense product template retrieval followed by candidate set listwise ranking. Stage 1 uses contrastive pretraining to learn a shared embedding space between products and reaction templates. Stage 2 refines template ranking over mined hard negative candidate sets with a multi positive listwise objective. To enable template side adaptation without destabilizing hard negative mining, ConRetroBert uses a slow moving exponential moving average template encoder for retrieval bank construction while updating the live template encoder through the ranking loss. On the local USPTO-50k benchmark, Stage 2 candidate set ranking improves top-1 reaction accuracy from 50.5% to 61.3%, while EMA stabilized template adaptation further improves it to 62.4%. Fine tuning from a leakage controlled USPTO-Full checkpoint reaches 75.4% top-1 accuracy on USPTO-50k. We also show that retrieval based template prediction is strong in the long tail of rare templates, and that many correct reactant predictions arise from alternative explicit templates rather than only the recorded positive label. Code and data are available at https://github.com/JahidBasher/ConRetroBert.