First Place Solution of KDD Cup 2021 & OGB Large-Scale Challenge Graph Prediction Track
Not on the map. Why?
Abstract (arXiv)
In this technical report, we present our solution of KDD Cup 2021 OGB Large-Scale Challenge - PCQM4M-LSC Track. We adopt Graphormer and ExpC as our basic models. We train each model by 8-fold cross-validation, and additionally train two Graphormer models on the union of training and validation sets with different random seeds. For final submission, we use a naive ensemble for these 18 models by taking average of their outputs. Using our method, our team MachineLearning achieved 0.1200 MAE on test set, which won the first place in KDD Cup graph prediction track.
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 10 of its 15 references that are in the map. First posted on arXiv 15 Jun 2021.
Builds on
10 of 15 references.
References · 10
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2018How Powerful are Graph Neural Networks?cited by 4,355
- 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
- 2017Deep Setscited by 1,394
- 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 875
- 2020Principal Neighbourhood Aggregation for Graph Netscited by 477
- 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
- 2021Do Transformers Really Perform Bad for Graph Representation?cited by 318
- 2020Breaking the Expressive Bottlenecks of Graph Neural Networkscited by 6
Built on by
Citing papers per year: 2021: 2; 2022: 3; 2023: 2; 2024: 3.
10 papers.
- 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
- 2023Rethinking the Expressive Power of GNNs via Graph Biconnectivitycited by 119
- 2024Scientific Large Language Models: A Survey on Biological & Chemical Domainscited by 80
- 2022Your Transformer May Not be as Powerful as You Expectcited by 55
- 2024Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Productscited by 35
- 2021Molecule3D: A Benchmark for Predicting 3D Geometries from Molecular Graphscited by 31
- 2022GPS++: An Optimised Hybrid MPNN/Transformer for Molecular Property Predictioncited by 28
- 2021LiteGEM: Lite Geometry Enhanced Molecular Representation Learning for Quantum Property Predictioncited by 6
- 2024GeoMFormer: A General Architecture for Geometric Molecular Representation Learningcited by 6
- 2023GPS++: Reviving the Art of Message Passing for Molecular Property Predictioncited by 5
- 2024GeoMFormer: A General Architecture for Geometric Molecular Representation Learning24 Jun 2024
- 2024Scientific Large Language Models: A Survey on Biological & Chemical Domains26 Jan 2024
- 2024Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Products18 Jan 2024
- 2023GPS++: Reviving the Art of Message Passing for Molecular Property Prediction6 Feb 2023
- 2023Rethinking the Expressive Power of GNNs via Graph Biconnectivity23 Jan 2023
- 2022GPS++: An Optimised Hybrid MPNN/Transformer for Molecular Property Prediction18 Nov 2022
- 2022Your Transformer May Not be as Powerful as You Expect26 May 2022
- 2022Recipe for a General, Powerful, Scalable Graph Transformer25 May 2022
- 2021Molecule3D: A Benchmark for Predicting 3D Geometries from Molecular Graphs30 Sep 2021
- 2021LiteGEM: Lite Geometry Enhanced Molecular Representation Learning for Quantum Property Prediction28 Jun 2021