mlsanityData to 14 Oct 2026

On the Equivalence of Graph Convolution and Mixup

arXiv 2310.00183 ↗First posted 29 Sep 2023cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper investigates the relationship between graph convolution and Mixup techniques. Graph convolution in a graph neural network involves aggregating features from neighboring samples to learn representative features for a specific node or sample. On the other hand, Mixup is a data augmentation technique that generates new examples by averaging features and one-hot labels from multiple samples. One commonality between these techniques is their utilization of information from multiple samples to derive feature representation. This study aims to explore whether a connection exists between these two approaches. Our investigation reveals that, under two mild conditions, graph convolution can be viewed as a specialized form of Mixup that is applied during both the training and testing phases. The two conditions are: 1) textitHomophily Relabel - assigning the target node's label to all its neighbors, and 2) textitTest-Time Mixup - Mixup the feature during the test time. We establish this equivalence mathematically by demonstrating that graph convolution networks (GCN) and simplified graph convolution (SGC) can be expressed as a form of Mixup. We also empirically verify the equivalence by training an MLP using the two conditions to achieve comparable performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 62 of its 82 references that are in the map. First posted on arXiv 29 Sep 2023.

Builds on

62 of 82 references.

References · 62

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2017Graph Attention Networkscited by 8,923
  4. 2017Inductive Representation Learning on Large Graphscited by 6,320
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  6. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  7. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  8. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
  9. 2019Simplifying Graph Convolutional Networkscited by 1,477
  10. 2016Variational Graph Auto-Encoderscited by 1,382
  11. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 993
  12. 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 877
38 more
  1. 2020Simple and Deep Graph Convolutional Networkscited by 860
  2. 2019Graph Neural Networks for Social Recommendationcited by 714
  3. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  4. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 541
  5. 2021Beyond Low-frequency Information in Graph Convolutional Networkscited by 362
  6. 2022DiffDock: Diffusion Steps, Twists, and Turns for Molecular Dockingcited by 313
  7. 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 305
  8. 2020SIGN: Scalable Inception Graph Neural Networkscited by 284
  9. 2022Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphscited by 264
  10. 2019Revisiting Graph Neural Networks: All We Have is Low-Pass Filterscited by 261
  11. 2020Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixupcited by 261
  12. 2020Generalization and Representational Limits of Graph Neural Networkscited by 218
  13. 2020Combining Label Propagation and Simple Models Out-performs Graph Neural Networkscited by 202
  14. 2020How Does Mixup Help With Robustness and Generalization?cited by 193
  15. 2019Graph Neural Networks with convolutional ARMA filterscited by 185
  16. 2020Design Space for Graph Neural Networkscited by 182
  17. 2019Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernelscited by 161
  18. 2018MixUp as Locally Linear Out-Of-Manifold Regularizationcited by 158
  19. 2022Learning on Large-scale Text-attributed Graphs via Variational Inferencecited by 155
  20. 2021Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillationcited by 152
  21. 2022G-Mixup: Graph Data Augmentation for Graph Classificationcited by 146
  22. 2023Do We Really Need Complicated Model Architectures For Temporal Networks?cited by 143
  23. 2021Co-Mixup: Saliency Guided Joint Mixup with Supermodular Diversitycited by 128
  24. 2023Rethinking the Expressive Power of GNNs via Graph Biconnectivitycited by 119
  25. 2022Graph Attention Multi-Layer Perceptroncited by 83
  26. 2022Graph Neural Networks for Link Prediction with Subgraph Sketchingcited by 81
  27. 2021Graph-MLP: Node Classification without Message Passing in Graphcited by 73
  28. 2023LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendationcited by 72
  29. 2021When and How Mixup Improves Calibrationcited by 60
  30. 2022Does GNN Pretraining Help Molecular Representation?cited by 60
  31. 2021Graph Coarsening with Neural Networkscited by 55
  32. 2022Graph Neural Networks are Dynamic Programmerscited by 55
  33. 2019Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attackscited by 54
  34. 2021Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced trainingcited by 54
  35. 2019Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracycited by 49
  36. 2022Retrieval-based Controllable Molecule Generationcited by 44
  37. 2022Graph Neural Networks are Inherently Good Generalizers: Insights by Bridging GNNs and MLPscited by 42
  38. 2022On Representing Linear Programs by Graph Neural Networkscited by 39

50 most cited of 62 All 62 (JSON)

Built on by

Not cited yet.