mlsanityData to 7 Oct 2026

GPS++: Reviving the Art of Message Passing for Molecular Property Prediction

arXiv 2302.02947 ↗First posted 6 Feb 2023cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

We present GPS++, a hybrid Message Passing Neural Network / Graph Transformer model for molecular property prediction. Our model integrates a well-tuned local message passing component and biased global attention with other key ideas from prior literature to achieve state-of-the-art results on large-scale molecular dataset PCQM4Mv2. Through a thorough ablation study we highlight the impact of individual components and find that nearly all of the model's performance can be maintained without any use of global self-attention, showing that message passing is still a competitive approach for 3D molecular property prediction despite the recent dominance of graph transformers. We also find that our approach is significantly more accurate than prior art when 3D positional information is not available.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 53 of its 77 references that are in the map. First posted on arXiv 6 Feb 2023.

Builds on

53 of 77 references.

References · 53

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Layer Normalizationcited by 5,579
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  7. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  8. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  9. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  10. 2017Mixed Precision Trainingcited by 1,135
  11. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  12. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
38 more
  1. 2016Deep Networks with Stochastic Depthcited by 973
  2. 2021E(n) Equivariant Graph Neural Networkscited by 951
  3. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
  4. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
  5. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 701
  6. 2020A Generalization of Transformer Networks to Graphscited by 653
  7. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
  8. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  9. 2020Directional Message Passing for Molecular Graphscited by 556
  10. 2021Rethinking Graph Transformers with Spectral Attentioncited by 468
  11. 2021Equivariant message passing for the prediction of tensorial properties and molecular spectracited by 438
  12. 2017Residual Gated Graph ConvNetscited by 341
  13. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  14. 2021Graph Neural Networks with Learnable Structural and Positional Representationscited by 326
  15. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 318
  16. 2020Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Moleculescited by 254
  17. 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
  18. 2020GEOM: Energy-annotated molecular conformations for property prediction and molecular generationcited by 238
  19. 2021Geometric and Physical Quantities Improve E(3) Equivariant Message Passingcited by 225
  20. 2021ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Predictioncited by 204
  21. 2022Pure Transformers are Powerful Graph Learnerscited by 188
  22. 2022Graph neural networks for materials science and chemistrycited by 181
  23. 2022TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentialscited by 179
  24. 2021Spherical Message Passing for 3D Graph Networkscited by 166
  25. 20213D Infomax improves GNNs for Molecular Property Predictioncited by 146
  26. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  27. 2021Global Self-Attention as a Replacement for Graph Convolutioncited by 113
  28. 2020LieTransformer: Equivariant self-attention for Lie Groupscited by 100
  29. 2022The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalystscited by 95
  30. 2022Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networkscited by 94
  31. 2023Everything is Connected: Graph Neural Networkscited by 85
  32. 2022One Transformer Can Understand Both 2D & 3D Molecular Datacited by 77
  33. 2019Molecular Property Prediction: A Multilevel Quantum Interactions Modeling Perspectivecited by 73
  34. 2022GRPE: Relative Positional Encoding for Graph Transformercited by 67
  35. 2022Benchmarking Graphormer on Large-Scale Molecular Modeling Datasetscited by 63
  36. 2021Simple GNN Regularisation for 3D Molecular Property Prediction & Beyondcited by 44
  37. 2022Equivariant Graph Attention Networks for Molecular Property Predictioncited by 19
  38. 2020Self-Supervised Graph Transformer on Large-Scale Molecular Datacited by 17

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 3.

5 papers.

  1. 2023Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasetscited by 37
  2. 2023GraphGPT: Generative Pre-trained Graph Eulerian Transformercited by 15
  3. 2024Generative Active Learning for the Search of Small-molecule Protein Binderscited by 6
  4. 2024How Molecules Impact Cells: Unlocking Contrastive PhenoMolecular Retrievalcited by 6
  5. 2024Best of Both Worlds: Advantages of Hybrid Graph Sequence Modelscited by 6