# Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction prediction

arXiv: https://arxiv.org/abs/2110.09681 · HTML: https://mlsanity.com/paper/2110.09681/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 77 references that are in the map.
- First posted on arXiv 19 Oct 2021.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [Analyzing Learned Molecular Representations for Property Prediction](https://mlsanity.com/paper/1904.01561/) (2019)
- [Do Transformers Really Perform Bad for Graph Representation?](https://mlsanity.com/paper/2106.05234/) (2021)
- [OpenNMT: Open-source Toolkit for Neural Machine Translation](https://mlsanity.com/paper/1709.03815/) (2017)
- [Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Prediction](https://mlsanity.com/paper/1811.02633/) (2018)
- [Retrosynthesis Prediction with Conditional Graph Logic Network](https://mlsanity.com/paper/2001.01408/) (2020)
- [Predicting Organic Reaction Outcomes with Weisfeiler-Lehman Network](https://mlsanity.com/paper/1709.04555/) (2017)
- [Molecule Attention Transformer](https://mlsanity.com/paper/2002.08264/) (2020)
- [Molecule Edit Graph Attention Network: Modeling Chemical Reactions as Sequences of Graph Edits](https://mlsanity.com/paper/2006.15426/) (2020)
- [Retrosynthetic reaction prediction using neural sequence-to-sequence models](https://mlsanity.com/paper/1706.01643/) (2017)
- [A Graph to Graphs Framework for Retrosynthesis Prediction](https://mlsanity.com/paper/2003.12725/) (2020)
- [Graph Transformer for Graph-to-Sequence Learning](https://mlsanity.com/paper/1911.07470/) (2019)
- [RetroXpert: Decompose Retrosynthesis Prediction like a Chemist](https://mlsanity.com/paper/2011.02893/) (2020)
- [SMILES Enumeration as Data Augmentation for Neural Network Modeling of Molecules](https://mlsanity.com/paper/1703.07076/) (2017)
- [Graph Transformation Policy Network for Chemical Reaction Prediction](https://mlsanity.com/paper/1812.09441/) (2018)
- ["Found in Translation": Predicting Outcomes of Complex Organic Chemistry Reactions using Neural Sequence-to-Sequence Models](https://mlsanity.com/paper/1711.04810/) (2017)
- [Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks](https://mlsanity.com/paper/1804.00823/) (2018)
- [A Generative Model For Electron Paths](https://mlsanity.com/paper/1805.10970/) (2018)

## Built on by (most cited first, 30 of 68)

- [RetroBridge: Modeling Retrosynthesis with Markov Bridges](https://mlsanity.com/paper/2308.16212/) (2023)
- [Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Model](https://mlsanity.com/paper/2211.10590/) (2022)
- [PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomes](https://mlsanity.com/paper/2406.13193/) (2024)
- [RetroComposer: Composing Templates for Template-Based Retrosynthesis Prediction](https://mlsanity.com/paper/2112.11225/) (2021)
- [Transformers and Large Language Models for Chemistry and Drug Discovery](https://mlsanity.com/paper/2310.06083/) (2023)
- [Electron flow matching for generative reaction mechanism prediction obeying conservation laws](https://mlsanity.com/paper/2502.12979/) (2025)
- [Synthelite: Chemist-aligned and feasibility-aware synthesis planning with LLMs](https://mlsanity.com/paper/2512.16424/) (2025)
- [MechRetro is a chemical-mechanism-driven graph learning framework for interpretable retrosynthesis prediction and pathway planning](https://mlsanity.com/paper/2210.02630/) (2022)
- [Node-Aligned Graph-to-Graph (NAG2G): Elevating Template-Free Deep Learning Approaches in Single-Step Retrosynthesis](https://mlsanity.com/paper/2309.15798/) (2023)
- [BatGPT-Chem: A Foundation Large Model For Retrosynthesis Prediction](https://mlsanity.com/paper/2408.10285/) (2024)
- [Enhancing Chemical Reaction and Retrosynthesis Prediction with Large Language Model and Dual-task Learning](https://mlsanity.com/paper/2505.02639/) (2025)
- [Retroformer: Pushing the Limits of Interpretable End-to-end Retrosynthesis Transformer](https://mlsanity.com/paper/2201.12475/) (2022)
- [Equivariant Denoisers Cannot Copy Graphs: Align Your Graph Diffusion Models](https://mlsanity.com/paper/2405.17656/) (2024)
- [Mind the Retrosynthesis Gap: Bridging the divide between Single-step and Multi-step Retrosynthesis Prediction](https://mlsanity.com/paper/2212.11809/) (2022)
- [RetroDiff: Retrosynthesis as Multi-stage Distribution Interpolation](https://mlsanity.com/paper/2311.14077/) (2023)
- [Tango*: Constrained synthesis planning using chemically informed value functions](https://mlsanity.com/paper/2412.03424/) (2024)
- [Chemist-aligned retrosynthesis by ensembling diverse inductive bias models](https://mlsanity.com/paper/2412.05269/) (2024)
- [Doubly Stochastic Graph-based Non-autoregressive Reaction Prediction](https://mlsanity.com/paper/2306.06119/) (2023)
- [UAlign: Pushing the Limit of Template-free Retrosynthesis Prediction with Unsupervised SMILES Alignment](https://mlsanity.com/paper/2404.00044/) (2024)
- [Challenging reaction prediction models to generalize to novel chemistry](https://mlsanity.com/paper/2501.06669/) (2025)
- [TempRe: Template generation for single and direct multi-step retrosynthesis](https://mlsanity.com/paper/2507.21762/) (2025)
- [AI for Interpretable Chemistry: Predicting Radical Mechanistic Pathways via Contrastive Learning](https://mlsanity.com/paper/2311.01118/) (2023)
- [Retrosynthesis prediction enhanced by in-silico reaction data augmentation](https://mlsanity.com/paper/2402.00086/) (2024)
- [Tokenization for Molecular Foundation Models](https://mlsanity.com/paper/2409.15370/) (2024)
- [Graph-to-SFILES: Control structure prediction from process topologies using generative artificial intelligence](https://mlsanity.com/paper/2412.00508/) (2024)
- [mathsfG2Retro as a Two-Step Graph Generative Models for Retrosynthesis Prediction](https://mlsanity.com/paper/2206.04882/) (2022)
- [A Unified View of Deep Learning for Reaction and Retrosynthesis Prediction: Current Status and Future Challenges](https://mlsanity.com/paper/2306.15890/) (2023)
- [RLSynC: Offline-Online Reinforcement Learning for Synthon Completion](https://mlsanity.com/paper/2309.02671/) (2023)
- [Holistic chemical evaluation reveals pitfalls in reaction prediction models](https://mlsanity.com/paper/2312.09004/) (2023)
- [Modeling Diverse Chemical Reactions for Single-step Retrosynthesis via Discrete Latent Variables](https://mlsanity.com/paper/2208.05482/) (2022)

## Abstract (arXiv)

Synthesis planning and reaction outcome prediction are two fundamental problems in computer-aided organic chemistry for which a variety of data-driven approaches have emerged. Natural language approaches that model each problem as a SMILES-to-SMILES translation lead to a simple end-to-end formulation, reduce the need for data preprocessing, and enable the use of well-optimized machine translation model architectures. However, SMILES representations are not an efficient representation for capturing information about molecular structures, as evidenced by the success of SMILES augmentation to boost empirical performance. Here, we describe a novel Graph2SMILES model that combines the power of Transformer models for text generation with the permutation invariance of molecular graph encoders that mitigates the need for input data augmentation. As an end-to-end architecture, Graph2SMILES can be used as a drop-in replacement for the Transformer in any task involving molecule(s)-to-molecule(s) transformations. In our encoder, an attention-augmented directed message passing neural network (D-MPNN) captures local chemical environments, and the global attention encoder allows for long-range and intermolecular interactions, enhanced by graph-aware positional embedding. Graph2SMILES improves the top-1 accuracy of the Transformer baselines by 1.7% and 1.9% for reaction outcome prediction on USPTO480k and USPTOSTEREO datasets respectively, and by 9.8% for one-step retrosynthesis on the USPTO50k dataset.