mlsanityData to 7 Oct 2026

Bag of Tricks for Node Classification with Graph Neural Networks

arXiv 2103.13355 ↗First posted 24 Mar 2021cs.LG cs.AIcited by 40

Not on the map. Why?

Abstract (arXiv)

Over the past few years, graph neural networks (GNN) and label propagation-based methods have made significant progress in addressing node classification tasks on graphs. However, in addition to their reliance on elaborate architectures and algorithms, there are several key technical details that are frequently overlooked, and yet nonetheless can play a vital role in achieving satisfactory performance. In this paper, we first summarize a series of existing tricks-of-the-trade, and then propose several new ones related to label usage, loss function formulation, and model design that can significantly improve various GNN architectures. We empirically evaluate their impact on final node classification accuracy by conducting ablation studies and demonstrate consistently-improved performance, often to an extent that outweighs the gains from more dramatic changes in the underlying GNN architecture. Notably, many of the top-ranked models on the Open Graph Benchmark (OGB) leaderboard and KDDCUP 2021 Large-Scale Challenge MAG240M-LSC benefit from these techniques we initiated.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 23 of its 32 references that are in the map. First posted on arXiv 24 Mar 2021.

Builds on

23 of 32 references.

References · 23

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2017Inductive Representation Learning on Large Graphscited by 6,284
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  6. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  7. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  8. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  9. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
  10. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  11. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
  12. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
11 more
  1. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  2. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 583
  3. 2020Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classificationcited by 500
  4. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  5. 2019Adversarial Examples Improve Image Recognitioncited by 283
  6. 2020SIGN: Scalable Inception Graph Neural Networkscited by 283
  7. 2020DeeperGCN: All You Need to Train Deeper GCNscited by 249
  8. 2020Combining Label Propagation and Simple Models Out-performs Graph Neural Networkscited by 199
  9. 2019Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networkscited by 177
  10. 2019Relational Graph Attention Networkscited by 81
  11. 2021Graph Neural Networks Inspired by Classical Iterative Algorithmscited by 60

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 7; 2023: 10; 2024: 9; 2025: 2; 2026: 3.

40 papers.

Sort:

By area:

  1. 2023Language is All a Graph Needscited by 219
  2. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  3. 2022Trustworthy Graph Neural Networks: Aspects, Methods and Trendscited by 88
  4. 2024Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classificationcited by 83
  5. 2022Graph Attention Multi-Layer Perceptroncited by 82
  6. 2021Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Studycited by 36
  7. 2023Towards Deep Attention in Graph Neural Networks: Problems and Remediescited by 24
  8. 2025Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellencecited by 24
  9. 2023Over-Squashing in Graph Neural Networks: A Comprehensive surveycited by 23
  10. 2023Label Information Enhanced Fraud Detection against Low Homophily in Graphscited by 22
  11. 2023Efficiently Forgetting What You Have Learned in Graph Representation Learning via Projectioncited by 20
  12. 2021Large-scale graph representation learning with very deep GNNs and self-supervisioncited by 18
28 more
  1. 2022Affinity-Aware Graph Networkscited by 17
  2. 2024Graph Sparsification via Mixture of Graphscited by 17
  3. 2023LazyGNN: Large-Scale Graph Neural Networks via Lazy Propagationcited by 16
  4. 2023Which Modality should I use -- Text, Motif, or Image? : Understanding Graphs with Large Language Modelscited by 14
  5. 2021Why Propagate Alone? Parallel Use of Labels and Features on Graphscited by 9
  6. 2021Does your graph need a confidence boost? Convergent boosted smoothing on graphs with tabular node featurescited by 9
  7. 2021Network In Graph Neural Networkcited by 8
  8. 2022Learnable Graph Convolutional Attention Networkscited by 8
  9. 2024Design Your Own Universe: A Physics-Informed Agnostic Method for Enhancing Graph Neural Networkscited by 6
  10. 2023Clarify Confused Nodes via Separated Learningcited by 5
  11. 2024How Does Message Passing Improve Collaborative Filtering?cited by 5
  12. 2024Training-free Graph Neural Networks and the Power of Labels as Featurescited by 5
  13. 2024Enhancing the Influence of Labels on Unlabeled Nodes in Graph Convolutional Networkscited by 4
  14. 2021Residual Network and Embedding Usage: New Tricks of Node Classification with Graph Convolutional Networkscited by 3
  15. 2021Graph Attention MLP with Reliable Label Utilizationcited by 3
  16. 2024Contextualized Messages Boost Graph Representationscited by 3
  17. 2022Predicting Corporate Risk by Jointly Modeling Company Networks and Dialogues in Earnings Conference Callscited by 2
  18. 2021AutoHEnsGNN: Winning Solution to AutoGraph Challenge for KDD Cup 2020cited by 1
  19. 2022A Robust Stacking Framework for Training Deep Graph Models with Multifaceted Node Featurescited by 1
  20. 2022Refined Edge Usage of Graph Neural Networks for Edge Predictioncited by 1
  21. 2024Heterogeneous Graph Neural Network on Semantic Treecited by 1
  22. 2023UniKG: A Benchmark and Universal Embedding for Large-Scale Knowledge Graphscited by 0
  23. 2023Model Stealing Attack against Graph Classification with Authenticity, Uncertainty and Diversitycited by 0
  24. 2024SFR-GNN: Simple and Fast Robust GNNs against Structural Attackscited by 0
  25. 2025ReDiSC: A Reparameterized Masked Diffusion Model for Scalable Node Classification with Structured Predictionscited by 0
  26. 2026Rethinking Multi-Label Node Classification: Do Tuned Classic GNNs Suffice?cited by 0
  27. 2026Revisiting Pre-Propagation GNNs: Robust Diffusion Operators and Hidden-State Re-Propagationcited by 0
  28. 2026InvWeaver: Deductive Feedback for Invariant Synthesis in Interacting-Loop Programscited by 0