mlsanityData to 7 Oct 2026

GPS++: An Optimised Hybrid MPNN/Transformer for Molecular Property Prediction

arXiv 2212.02229 ↗First posted 18 Nov 2022q-bio.QM cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

This technical report presents GPS++, the first-place solution to the Open Graph Benchmark Large-Scale Challenge (OGB-LSC 2022) for the PCQM4Mv2 molecular property prediction task. Our approach implements several key principles from the prior literature. At its core our GPS++ method is a hybrid MPNN/Transformer model that incorporates 3D atom positions and an auxiliary denoising task. The effectiveness of GPS++ is demonstrated by achieving 0.0719 mean absolute error on the independent test-challenge PCQM4Mv2 split. Thanks to Graphcore IPU acceleration, GPS++ scales to deep architectures (16 layers), training at 3 minutes per epoch, and large ensemble (112 models), completing the final predictions in 1 hour 32 minutes, well under the 4 hour inference budget allocated. Our implementation is publicly available at: https://github.com/graphcore/ogb-lsc-pcqm4mv2.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 23 of its 36 references that are in the map. First posted on arXiv 18 Nov 2022.

Builds on

23 of 36 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  4. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  5. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  6. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  7. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  8. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  9. 2017Mixed Precision Trainingcited by 1,135
  10. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  11. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
  12. 2020A Generalization of Transformer Networks to Graphscited by 653
11 more
  1. 2021Rethinking Graph Transformers with Spectral Attentioncited by 468
  2. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  3. 2021Graph Neural Networks with Learnable Structural and Positional Representationscited by 326
  4. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  5. 2021Global Self-Attention as a Replacement for Graph Convolutioncited by 113
  6. 2022Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networkscited by 94
  7. 2022One Transformer Can Understand Both 2D & 3D Molecular Datacited by 77
  8. 2022GRPE: Relative Positional Encoding for Graph Transformercited by 67
  9. 2022Benchmarking Graphormer on Large-Scale Molecular Modeling Datasetscited by 63
  10. 2021First Place Solution of KDD Cup 2021 & OGB Large-Scale Challenge Graph Prediction Trackcited by 10
  11. 2022Reducing Down(stream)time: Pretraining Molecular GNNs using Heterogeneous AI Acceleratorscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 10; 2025: 3; 2026: 6.

28 papers.

Sort:

  1. 2023Graph Inductive Biases in Transformers without Message Passingcited by 150
  2. 2024Position: Graph Foundation Models are Already Herecited by 84
  3. 2023Attending to Graph Transformerscited by 81
  4. 2024On the Scalability of GNNs for Molecular Graphscited by 36
  5. 2023Automated 3D Pre-Training for Molecular Property Predictioncited by 24
  6. 2023Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+cited by 22
  7. 2023PubChemQC B3LYP/6-31G*//PM6 dataset: the Electronic Structures of 86 Million Molecules using B3LYP/6-31G* calculationscited by 12
  8. 2024Triplet Interaction Improves Graph Transformers: Accurate Molecular Graph Learning with Triplet Graph Transformerscited by 12
  9. 2024Geometry Informed Tokenization of Molecules for Language Model Generationcited by 10
  10. 2025Platonic Transformers: A Solid Choice For Equivariancecited by 9
  11. 2024Generalizing Graph Transformers Across Diverse Graphs and Tasks via Pre-trainingcited by 7
  12. 2023Union Subgraph Neural Networkscited by 6
16 more
  1. 2024Impact of Domain Knowledge and Multi-Modality on Intelligent Molecular Property Prediction: A Systematic Surveycited by 6
  2. 2024GeoMFormer: A General Architecture for Geometric Molecular Representation Learningcited by 6
  3. 2025CellCLIP -- Learning Perturbation Effects in Cell Painting via Text-Guided Contrastive Learningcited by 6
  4. 2023GeoTMI:Predicting quantum chemical property with easy-to-obtain geometry via positional denoisingcited by 5
  5. 2025Structural Priors and Modular Adapters in the Composable Fine-Tuning Algorithm of Large-Scale Modelscited by 4
  6. 2026Enhancing Molecular Property Predictions by Learning from Bond Modelling and Interactionscited by 3
  7. 2024Differentiable Cluster Graph Neural Networkcited by 2
  8. 2026A Systematic Survey and Benchmark of Deep Learning for Molecular Property Prediction in the Foundation Model Eracited by 1
  9. 2023Is Self-Supervised Pretraining Good for Extrapolation in Molecular Property Prediction?cited by 0
  10. 2023Generating QM1B with PySCFIPUcited by 0
  11. 2024Efficient Training of Transformers for Molecule Property Prediction on Small-scale Datasetscited by 0
  12. 2024M3-20M: A Large-Scale Multi-Modal Molecule Dataset for AI-driven Drug Design and Discoverycited by 0
  13. 2026AdvSynGNN: Structure-Adaptive Graph Neural Nets via Adversarial Synthesis and Self-Corrective Propagationcited by 0
  14. 2026MolFM-Lite: Multi-Modal Molecular Property Prediction with Conformer Ensemble Attention and Cross-Modal Fusioncited by 0
  15. 2026k-Maximum Inner Product Attention for Graph Transformers and the Expressive Power of GraphGPScited by 0
  16. 2026Probabilistic Contrastive Pretraining for Multi-task ADME Property Predictioncited by 0