mlsanityData to 7 Oct 2026

Benchmarking Graphormer on Large-Scale Molecular Modeling Datasets

arXiv 2203.04810 ↗First posted 9 Mar 2022cs.LGcited by 63

Not on the map. Why?

Abstract (arXiv)

This technical note describes the recent updates of Graphormer, including architecture design modifications, and the adaption to 3D molecular dynamics simulation. With these simple modifications, Graphormer could attain better results on large-scale molecular modeling datasets than the vanilla one, and the performance gain could be consistently obtained on 2D and 3D molecular graph modeling tasks. In addition, we show that with a global receptive field and an adaptive aggregation strategy, Graphormer is more powerful than classic message-passing-based GNNs. Empirically, Graphormer could achieve much less MAE than the originally reported results on the PCQM4M quantum chemistry dataset used in KDD Cup 2021. In the meanwhile, it greatly outperforms the competitors in the recent Open Catalyst Challenge, which is a competition track on NeurIPS 2021 workshop, and aims to model the catalyst-adsorbate reaction system with advanced AI models. All codes could be found at https://github.com/Microsoft/Graphormer.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 16 of its 24 references that are in the map. First posted on arXiv 9 Mar 2022.

Builds on

16 of 24 references.

References · 16

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Inductive Representation Learning on Large Graphscited by 6,284
  3. 2018How Powerful are Graph Neural Networks?cited by 4,355
  4. 2020On Layer Normalization in the Transformer Architecturecited by 906
  5. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 878
  6. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  7. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 318
  8. 2020The Open Catalyst 2020 (OC20) Dataset and Community Challengescited by 246
  9. 2019What graph neural networks cannot learn: depth vs widthcited by 195
  10. 2020The Surprising Power of Graph Neural Networks with Random Node Initializationcited by 190
  11. 2020Random Features Strengthen Graph Neural Networkscited by 184
  12. 2021Breaking the Limits of Message Passing Graph Neural Networkscited by 101
4 more
  1. 2021Rotation Invariant Graph Neural Networks using Spin Convolutionscited by 52
  2. 2020An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storagecited by 45
  3. 2015Algebraic Methods in the Congested Cliquecited by 2
  4. 2017MST in O(1) Rounds of the Congested Cliquecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 22; 2024: 18; 2025: 12; 2026: 3.

63 papers.

Sort:

By area:

  1. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
  2. 2022Long Range Graph Benchmarkcited by 258
  3. 2022Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphscited by 258
  4. 2023EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representationscited by 201
  5. 2022Pure Transformers are Powerful Graph Learnerscited by 188
  6. 2023Rethinking the Expressive Power of GNNs via Graph Biconnectivitycited by 119
  7. 2023On the Expressive Power of Geometric Graph Neural Networkscited by 89
  8. 2023Attending to Graph Transformerscited by 81
  9. 2022One Transformer Can Understand Both 2D & 3D Molecular Datacited by 77
  10. 2023On the Connection Between MPNN and Graph Transformercited by 60
  11. 2022Your Transformer May Not be as Powerful as You Expectcited by 55
  12. 2025Position: Graph Learning Will Lose Relevance Due To Poor Benchmarkscited by 52
38 more
  1. 2024Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Productscited by 35
  2. 2022GPS++: An Optimised Hybrid MPNN/Transformer for Molecular Property Predictioncited by 28
  3. 2024A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applicationscited by 28
  4. 2023Learning Joint 2D & 3D Diffusion Models for Complete Molecule Generationcited by 27
  5. 2023Towards Predicting Equilibrium Distributions for Molecular Systems with Deep Learningcited by 26
  6. 2023Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+cited by 22
  7. 2023Exposition on over-squashing problem on GNNs: Current Methods, Benchmarks and Challengescited by 22
  8. 2024Uni-Mol2: Exploring Molecular Pretraining Model at Scalecited by 20
  9. 2023GraphGPT: Generative Pre-trained Graph Eulerian Transformercited by 15
  10. 2024Triplet Interaction Improves Graph Transformers: Accurate Molecular Graph Learning with Triplet Graph Transformerscited by 12
  11. 2023Overcoming the Barrier of Orbital-Free Density Functional Theory for Molecular Systems Using Deep Learningcited by 11
  12. 2024An end-to-end attention-based approach for learning on graphscited by 11
  13. 2023Learning a Fourier Transform for Linear Relative Positional Encodings in Transformerscited by 10
  14. 2023Lo-Hi: Practical ML Drug Discovery Benchmarkcited by 10
  15. 2024Geometry Informed Tokenization of Molecules for Language Model Generationcited by 10
  16. 2023Graph Propagation Transformer for Graph Representation Learningcited by 9
  17. 2023Learning Large Graph Property Prediction via Graph Segment Trainingcited by 9
  18. 2024FlowerFormer: Empowering Neural Architecture Encoding using a Flow-aware Graph Transformercited by 9
  19. 2024Pre-Training Protein Bi-level Representation Through Span Mask Strategy On 3D Protein Chainscited by 6
  20. 2024GeoMFormer: A General Architecture for Geometric Molecular Representation Learningcited by 6
  21. 2023GPS++: Reviving the Art of Message Passing for Molecular Property Predictioncited by 5
  22. 2023Diffusing Graph Attentioncited by 5
  23. 2025Synergistic Benefits of Joint Molecule Generation and Property Predictioncited by 5
  24. 2023Molecular Geometry-aware Transformer for accurate 3D Atomic System modelingcited by 4
  25. 2023DR-Label: Improving GNN Models for Catalysis Systems by Label Deconstruction and Reconstructioncited by 4
  26. 2023iLoRE: Dynamic Graph Representation with Instant Long-term Modeling and Re-occurrence Preservationcited by 4
  27. 2024Structure-Enhanced Protein Instruction Tuning: Towards General-Purpose Protein Understanding with LLMscited by 4
  28. 2022AutoML for Climate Change: A Call to Actioncited by 3
  29. 20233D Molecular Generation via Virtual Dynamicscited by 3
  30. 2023Learning to Group Auxiliary Datasets for Moleculecited by 3
  31. 2024Molecule Graph Networks with Many-body Equivariant Interactionscited by 3
  32. 2024DTFormer: A Transformer-Based Method for Discrete-Time Dynamic Graph Representation Learningcited by 3
  33. 2024Physical Consistency Bridges Heterogeneous Data in Molecular Multi-Task Learningcited by 3
  34. 2025InertialAR: Autoregressive 3D Molecule Generation with Inertial Framescited by 3
  35. 2026Enhancing Molecular Property Predictions by Learning from Bond Modelling and Interactionscited by 3
  36. 2024nabla2DFT: A Universal Quantum Chemistry Dataset of Drug-Like Molecules and a Benchmark for Neural Network Potentialscited by 2
  37. 2024MVGT: A Multi-view Graph Transformer Based on Spatial Relations for EEG Emotion Recognitioncited by 2
  38. 2024Bridging Geometric States via Geometric Diffusion Bridgecited by 2

50 most cited of 63 All 63 (JSON)