mlsanityData to 7 Oct 2026

SCR: Training Graph Neural Networks with Consistency Regularization

arXiv 2112.04319 ↗First posted 8 Dec 2021cs.SI cs.LG stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

We present the SCR framework for enhancing the training of graph neural networks (GNNs) with consistency regularization. Regularization is a set of strategies used in Machine Learning to reduce overfitting and improve the generalization ability. However, it is unclear how to best design the generalization strategies in GNNs, as it works in a semi-supervised setting for graph data. The major challenge lies in how to efficiently balance the trade-off between the error from the labeled data and that from the unlabeled data. SCR is a simple yet general framework in which we introduce two strategies of consistency regularization to address the challenge above. One is to minimize the disagreements among the perturbed predictions by different versions of a GNN model. The other is to leverage the Mean Teacher paradigm to estimate a consistency loss between teacher and student models instead of the disagreement of the predictions. We conducted experiments on three large-scale node classification datasets in the Open Graph Benchmark (OGB). Experimental results demonstrate that the proposed SCR framework is a general one that can enhance various GNNs to achieve better performance. Finally, SCR has been the top-1 entry on all three OGB leaderboards as of this submission.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 36 of its 51 references that are in the map. First posted on arXiv 8 Dec 2021.

Builds on

36 of 51 references.

References · 36

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2018How Powerful are Graph Neural Networks?cited by 4,355
  7. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  8. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  9. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  10. 2019Simplifying Graph Convolutional Networkscited by 1,470
  11. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  12. 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,134
24 more
  1. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
  2. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  3. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  4. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  5. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 583
  6. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 540
  7. 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 470
  8. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  9. 2018Invariant and Equivariant Graph Networkscited by 339
  10. 2020Towards Deeper Graph Neural Networkscited by 296
  11. 2020SIGN: Scalable Inception Graph Neural Networkscited by 283
  12. 2014Learning with Pseudo-Ensemblescited by 241
  13. 2020Graph Random Neural Network for Semi-Supervised Learning on Graphscited by 213
  14. 2021R-Drop: Regularized Dropout for Neural Networkscited by 202
  15. 2020Combining Label Propagation and Simple Models Out-performs Graph Neural Networkscited by 199
  16. 2018Adaptive Sampling Towards Fast Graph Representation Learningcited by 180
  17. 2021Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Predictioncited by 137
  18. 2019Multi-Stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labelscited by 111
  19. 2019Graph Adversarial Training: Dynamically Regularizing Based on Graph Structurecited by 93
  20. 2021Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced trainingcited by 54
  21. 2020PECOS: Prediction for Enormous and Correlated Output Spacescited by 50
  22. 2019Batch Virtual Adversarial Training for Graph Convolutional Networkscited by 33
  23. 2020Scalable Graph Neural Networks for Heterogeneous Graphscited by 31
  24. 2020Extreme Multi-label Classification from Aggregated Labelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 3; 2025: 3; 2026: 1.

10 papers.

  1. 2024UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphscited by 60
  2. 2025UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphscited by 47
  3. 2025Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?cited by 26
  4. 2023LazyGNN: Large-Scale Graph Neural Networks via Lazy Propagationcited by 16
  5. 2023Efficient Heterogeneous Graph Learning via Random Projectioncited by 16
  6. 2024Generalizing Graph Transformers Across Diverse Graphs and Tasks via Pre-trainingcited by 7
  7. 2023Graph-based Knowledge Distillation: A survey and experimental evaluationcited by 6
  8. 2025Integrating Structural and Semantic Signals in Text-Attributed Graphs with BiGTexcited by 3
  9. 2024Variational Graph Auto-Encoder Based Inductive Learning Method for Semi-Supervised Classificationcited by 1
  10. 2026Optimal Transport-Guided Adversarial Attacks on Graph Neural Network-Based Bot Detectioncited by 1