# MassFormer: Tandem Mass Spectrum Prediction for Small Molecules using Graph Transformers

arXiv: https://arxiv.org/abs/2111.04824 · HTML: https://mlsanity.com/paper/2111.04824/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 111 references that are in the map.
- First posted on arXiv 8 Nov 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View](https://mlsanity.com/paper/1909.03211/) (2019)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs](https://mlsanity.com/paper/2103.09430/) (2021)
- [Do Transformers Really Perform Bad for Graph Representation?](https://mlsanity.com/paper/2106.05234/) (2021)
- [Towards Deeper Graph Neural Networks](https://mlsanity.com/paper/2007.09296/) (2020)
- [Predicting Organic Reaction Outcomes with Weisfeiler-Lehman Network](https://mlsanity.com/paper/1709.04555/) (2017)
- [Molecule Attention Transformer](https://mlsanity.com/paper/2002.08264/) (2020)
- [BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance](https://mlsanity.com/paper/1911.02969/) (2019)
- [Memory-Based Graph Networks](https://mlsanity.com/paper/2002.09518/) (2020)
- [DGL-LifeSci: An Open-Source Toolkit for Deep Learning on Graphs in Life Science](https://mlsanity.com/paper/2106.14232/) (2021)
- [Rapid Prediction of Electron-Ionization Mass Spectrometry using Neural Networks](https://mlsanity.com/paper/1811.08545/) (2018)
- [The Curse of Performance Instability in Analysis Datasets: Consequences, Source, and Suggestions](https://mlsanity.com/paper/2004.13606/) (2020)
- [Neural Machine Translation: A Review of Methods, Resources, and Tools](https://mlsanity.com/paper/2012.15515/) (2020)
- [Using Graph Neural Networks for Mass Spectrometry Prediction](https://mlsanity.com/paper/2010.04661/) (2020)
- [Competitive Fragmentation Modeling of ESI-MS/MS spectra for putative metabolite identification](https://mlsanity.com/paper/1312.0264/) (2013)
- [Fragmentation trees reloaded](https://mlsanity.com/paper/1412.1929/) (2014)

## Built on by (most cited first, 2 of 2)

- [Artificial Intelligence in Spectroscopy: Advancing Chemistry from Prediction to Generation and Beyond](https://mlsanity.com/paper/2502.09897/) (2025)
- [FraGNNet: A Deep Probabilistic Model for Tandem Mass Spectrum Prediction](https://mlsanity.com/paper/2404.02360/) (2024)

## Abstract (arXiv)

Tandem mass spectra capture fragmentation patterns that provide key structural information about a molecule. Although mass spectrometry is applied in many areas, the vast majority of small molecules lack experimental reference spectra. For over seventy years, spectrum prediction has remained a key challenge in the field. Existing deep learning methods do not leverage global structure in the molecule, potentially resulting in difficulties when generalizing to new data. In this work we propose a new model, MassFormer, for accurately predicting tandem mass spectra. MassFormer uses a graph transformer architecture to model long-distance relationships between atoms in the molecule. The transformer module is initialized with parameters obtained through a chemical pre-training task, then fine-tuned on spectral data. MassFormer outperforms competing approaches for spectrum prediction on multiple datasets, and is able to recover prior knowledge about the effect of collision energy on the spectrum. By employing gradient-based attribution methods, we demonstrate that the model can identify relationships between fragment peaks. To further highlight MassFormer's utility, we show that it can match or exceed existing prediction-based methods on two spectrum identification tasks. We provide open-source implementations of our model and baseline approaches, with the goal of encouraging future research in this area.