mlsanityData to 7 Oct 2026

Triplet Interaction Improves Graph Transformers: Accurate Molecular Graph Learning with Triplet Graph Transformers

arXiv 2402.04538 ↗First posted 7 Feb 2024cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Graph transformers typically lack third-order interactions, limiting their geometric understanding which is crucial for tasks like molecular geometry prediction. We propose the Triplet Graph Transformer (TGT) that enables direct communication between pairs within a 3-tuple of nodes via novel triplet attention and aggregation mechanisms. TGT is applied to molecular property prediction by first predicting interatomic distances from 2D graphs and then using these distances for downstream tasks. A novel three-stage training procedure and stochastic inference further improve training efficiency and model performance. Our model achieves new state-of-the-art (SOTA) results on open challenge benchmarks PCQM4Mv2 and OC20 IS2RE. We also obtain SOTA results on QM9, MOLPCBA, and LIT-PCBA molecular property prediction benchmarks via transfer learning. We also demonstrate the generality of TGT with SOTA results on the traveling salesman problem (TSP).

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 60 of its 76 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

60 of 76 references.

References · 60

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2017Graph Attention Networkscited by 8,854
  6. 2016Layer Normalizationcited by 5,579
  7. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  8. 2018How Powerful are Graph Neural Networks?cited by 4,355
  9. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  10. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  11. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  12. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
38 more
  1. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  2. 2021E(n) Equivariant Graph Neural Networkscited by 951
  3. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
  4. 2020Benchmarking Graph Neural Networkscited by 672
  5. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  6. 2017SchNet: A continuous-filter convolutional neural network for modeling quantum interactionscited by 569
  7. 2020Directional Message Passing for Molecular Graphscited by 556
  8. 2020Principal Neighbourhood Aggregation for Graph Netscited by 477
  9. 2021Rethinking Graph Transformers with Spectral Attentioncited by 468
  10. 2021Equivariant message passing for the prediction of tensorial properties and molecular spectracited by 438
  11. 2021GemNet: Universal Directional Graph Neural Networks for Moleculescited by 342
  12. 2017Residual Gated Graph ConvNetscited by 341
  13. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  14. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 318
  15. 2019Cormorant: Covariant Molecular Neural Networkscited by 262
  16. 2020Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Moleculescited by 254
  17. 2020DeeperGCN: All You Need to Train Deeper GCNscited by 249
  18. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  19. 2020The Open Catalyst 2020 (OC20) Dataset and Community Challengescited by 246
  20. 2021Geometric and Physical Quantities Improve E(3) Equivariant Message Passingcited by 225
  21. 2021ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Predictioncited by 204
  22. 2022Representing Long-Range Context for Graph Neural Networks with Global Attentioncited by 193
  23. 2022Pure Transformers are Powerful Graph Learnerscited by 188
  24. 2022TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentialscited by 179
  25. 20213D Infomax improves GNNs for Molecular Property Predictioncited by 146
  26. 2020Directional Graph Networkscited by 132
  27. 2021Global Self-Attention as a Replacement for Graph Convolutioncited by 113
  28. 2023On the Expressive Power of Geometric Graph Neural Networkscited by 89
  29. 2022One Transformer Can Understand Both 2D & 3D Molecular Datacited by 77
  30. 2022GRPE: Relative Positional Encoding for Graph Transformercited by 67
  31. 2022Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matchingcited by 67
  32. 2022Benchmarking Graphormer on Large-Scale Molecular Modeling Datasetscited by 63
  33. 2021Simple GNN Regularisation for 3D Molecular Property Prediction & Beyondcited by 44
  34. 2020Graph convolutions that can finally model local structurecited by 31
  35. 2022FP-GNN: a versatile deep learning architecture for enhanced molecular property predictioncited by 31
  36. 2022GPS++: An Optimised Hybrid MPNN/Transformer for Molecular Property Predictioncited by 28
  37. 2023Automated 3D Pre-Training for Molecular Property Predictioncited by 24
  38. 2023Is Distance Matrix Enough for Geometric Deep Learning?cited by 22

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 4.

12 papers.

  1. 2025A Survey of Graph Transformers: Architectures, Theories and Applicationscited by 12
  2. 2024An end-to-end attention-based approach for learning on graphscited by 11
  3. 2025Graph-structured Small Molecule Drug Discovery Through Deep Learning: Progress, Challenges, and Opportunitiescited by 5
  4. 2025Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More Morecited by 3
  5. 2025GeoRecon: Graph-Level Representation Learning for 3D Molecules via Reconstruction-Based Pretrainingcited by 3
  6. 2026De Novo Molecular Generation from Mass Spectra via Many-Body Enhanced Diffusioncited by 3
  7. 2026Enhancing Molecular Property Predictions by Learning from Bond Modelling and Interactionscited by 3
  8. 2025Graph Alignment for Benchmarking Graph Neural Networks and Learning Positional Encodingscited by 1
  9. 2025Connectivity-Guided Sparsification of 2-FWL GNNs: Preserving Full Expressivity with Improved Efficiencycited by 0
  10. 2025How Smoothing is N-simplicial Attention?cited by 0
  11. 2026Unlocking High-Fidelity Molecular Generation from Mass Spectra via Dual-Stream Line Graph Diffusioncited by 0
  12. 2026Attention Graphons: A Graph Limit Perspective on Graph Transformerscited by 0