# ReactionT5: a large-scale pre-trained model towards application of limited reaction data

arXiv: https://arxiv.org/abs/2311.06708 · HTML: https://mlsanity.com/paper/2311.06708/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 12 Nov 2023.

## Builds on (most cited first, 10 of 10)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [Scaling Laws for Transfer](https://mlsanity.com/paper/2102.01293/) (2021)
- [Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Prediction](https://mlsanity.com/paper/1811.02633/) (2018)
- [Translation between Molecules and Natural Language](https://mlsanity.com/paper/2204.11817/) (2022)
- [Predicting Organic Reaction Outcomes with Weisfeiler-Lehman Network](https://mlsanity.com/paper/1709.04555/) (2017)
- [Unifying Molecular and Textual Representations via Multi-task Language Modelling](https://mlsanity.com/paper/2301.12586/) (2023)
- [A Systematic Survey of Chemical Pre-trained Models](https://mlsanity.com/paper/2210.16484/) (2022)
- [Deriving Neural Architectures from Sequence and Graph Kernels](https://mlsanity.com/paper/1705.09037/) (2017)

## Built on by (most cited first, 13 of 13)

- [A Review of Large Language Models and Autonomous Agents in Chemistry](https://mlsanity.com/paper/2407.01603/) (2024)
- [MolTC: Towards Molecular Relational Modeling In Language Models](https://mlsanity.com/paper/2402.03781/) (2024)
- [ChemDFM-X: Towards Large Multimodal Model for Chemistry](https://mlsanity.com/paper/2409.13194/) (2024)
- [MolX: Enhancing Large Language Models for Molecular Understanding With A Multi-Modal Extension](https://mlsanity.com/paper/2406.06777/) (2024)
- [Tokenization for Molecular Foundation Models](https://mlsanity.com/paper/2409.15370/) (2024)
- [FoodPuzzle: Developing Large Language Model Agents as Flavor Scientists](https://mlsanity.com/paper/2409.12832/) (2024)
- [Generative Design of Functional Metal Complexes Utilizing the Internal Knowledge of Large Language Models](https://mlsanity.com/paper/2410.18136/) (2024)
- [Specialising and Analysing Instruction-Tuned and Byte-Level Language Models for Organic Reaction Prediction](https://mlsanity.com/paper/2405.10625/) (2024)
- [Augmenting Molecular Language Models with Local n-gram Memory](https://mlsanity.com/paper/2606.12113/) (2026)
- [ModuLM: Enabling Modular and Multimodal Molecular Relational Learning with Large Language Models](https://mlsanity.com/paper/2506.00880/) (2025)
- [CovDocker: Benchmarking Covalent Drug Design with Tasks, Datasets, and Solutions](https://mlsanity.com/paper/2506.21085/) (2025)
- [Modular Multi-Task Learning for Chemical Reaction Prediction](https://mlsanity.com/paper/2602.10404/) (2026)
- [Where to cut, how deep: BPE and Unigram-LM on chemistry SMILES](https://mlsanity.com/paper/2607.05691/) (2026)

## Abstract (arXiv)

Transformer-based deep neural networks have revolutionized the field of molecular-related prediction tasks by treating molecules as symbolic sequences. These models have been successfully applied in various organic chemical applications by pretraining them with extensive compound libraries and subsequently fine-tuning them with smaller in-house datasets for specific tasks. However, many conventional methods primarily focus on single molecules, with limited exploration of pretraining for reactions involving multiple molecules. In this paper, we propose ReactionT5, a novel model that leverages pretraining on the Open Reaction Database (ORD), a publicly available large-scale resource. We further fine-tune this model for yield prediction and product prediction tasks, demonstrating its impressive performance even with limited fine-tuning data compared to traditional models. The pre-trained ReactionT5 model is publicly accessible on the Hugging Face platform.