# Retroformer: Pushing the Limits of Interpretable End-to-end Retrosynthesis Transformer

arXiv: https://arxiv.org/abs/2201.12475 · HTML: https://mlsanity.com/paper/2201.12475/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 29 Jan 2022.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [A Generalization of Transformer Networks to Graphs](https://mlsanity.com/paper/2012.09699/) (2020)
- [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017)
- [Do Transformers Really Perform Bad for Graph Representation?](https://mlsanity.com/paper/2106.05234/) (2021)
- [Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://mlsanity.com/paper/1610.02424/) (2016)
- [OpenNMT: Open-source Toolkit for Neural Machine Translation](https://mlsanity.com/paper/1709.03815/) (2017)
- [Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Prediction](https://mlsanity.com/paper/1811.02633/) (2018)
- [Retrosynthesis Prediction with Conditional Graph Logic Network](https://mlsanity.com/paper/2001.01408/) (2020)
- [Molecule Edit Graph Attention Network: Modeling Chemical Reactions as Sequences of Graph Edits](https://mlsanity.com/paper/2006.15426/) (2020)
- [A Graph to Graphs Framework for Retrosynthesis Prediction](https://mlsanity.com/paper/2003.12725/) (2020)
- [RetroXpert: Decompose Retrosynthesis Prediction like a Chemist](https://mlsanity.com/paper/2011.02893/) (2020)
- [Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction prediction](https://mlsanity.com/paper/2110.09681/) (2021)
- [Dual-view Molecule Pre-training](https://mlsanity.com/paper/2106.10234/) (2021)
- [Learning to Make Generalizable and Diverse Predictions for Retrosynthesis](https://mlsanity.com/paper/1910.09688/) (2019)
- [Relative Molecule Self-Attention Transformer](https://mlsanity.com/paper/2110.05841/) (2021)
- [Learning Attributed Graph Representations with Communicative Message Passing Transformer](https://mlsanity.com/paper/2107.08773/) (2021)
- [Guiding Attention for Self-Supervised Learning with Transformers](https://mlsanity.com/paper/2010.02399/) (2020)

## Built on by (most cited first, 8 of 8)

- [InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery](https://mlsanity.com/paper/2311.16208/) (2023)
- [Retrosynthesis Prediction with Local Template Retrieval](https://mlsanity.com/paper/2306.04123/) (2023)
- [Deep Learning Methods for Small Molecule Drug Discovery: A Survey](https://mlsanity.com/paper/2303.00313/) (2023)
- [MechRetro is a chemical-mechanism-driven graph learning framework for interpretable retrosynthesis prediction and pathway planning](https://mlsanity.com/paper/2210.02630/) (2022)
- [Node-Aligned Graph-to-Graph (NAG2G): Elevating Template-Free Deep Learning Approaches in Single-Step Retrosynthesis](https://mlsanity.com/paper/2309.15798/) (2023)
- [MotifRetro: Exploring the Combinability-Consistency Trade-offs in retrosynthesis via Dynamic Motif Editing](https://mlsanity.com/paper/2305.15153/) (2023)
- [ReactAIvate: A Deep Learning Approach to Predicting Reaction Mechanisms and Unmasking Reactivity Hotspots](https://mlsanity.com/paper/2407.10090/) (2024)
- [Variational-Ising-Attention:Tailored Attention Matters for Science](https://mlsanity.com/paper/2607.23634/) (2026)

## Abstract (arXiv)

Retrosynthesis prediction is one of the fundamental challenges in organic synthesis. The task is to predict the reactants given a core product. With the advancement of machine learning, computer-aided synthesis planning has gained increasing interest. Numerous methods were proposed to solve this problem with different levels of dependency on additional chemical knowledge. In this paper, we propose Retroformer, a novel Transformer-based architecture for retrosynthesis prediction without relying on any cheminformatics tools for molecule editing. Via the proposed local attention head, the model can jointly encode the molecular sequence and graph, and efficiently exchange information between the local reactive region and the global reaction context. Retroformer reaches the new state-of-the-art accuracy for the end-to-end template-free retrosynthesis, and improves over many strong baselines on better molecule and reaction validity. In addition, its generative procedure is highly interpretable and controllable. Overall, Retroformer pushes the limits of the reaction reasoning ability of deep generative models.