mlsanityData to 7 Oct 2026

Learning to Model Graph Structural Information on MLPs via Graph Structure Self-Contrasting

arXiv 2409.05573 ↗First posted 9 Sep 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent years have witnessed great success in handling graph-related tasks with Graph Neural Networks (GNNs). However, most existing GNNs are based on message passing to perform feature aggregation and transformation, where the structural information is explicitly involved in the forward propagation by coupling with node features through graph convolution at each layer. As a result, subtle feature noise or structure perturbation may cause severe error propagation, resulting in extremely poor robustness. In this paper, we rethink the roles played by graph structural information in graph data training and identify that message passing is not the only path to modeling structural information. Inspired by this, we propose a simple but effective Graph Structure Self-Contrasting (GSSC) framework that learns graph structural information without message passing. The proposed framework is based purely on Multi-Layer Perceptrons (MLPs), where the structural information is only implicitly incorporated as prior knowledge to guide the computation of supervision signals, substituting the explicit message propagation as in GNNs. Specifically, it first applies structural sparsification to remove potentially uninformative or noisy edges in the neighborhood, and then performs structural self-contrasting in the sparsified neighborhood to learn robust node representations. Finally, structural sparsification and self-contrasting are formulated as a bi-level optimization problem and solved in a unified framework. Extensive experiments have qualitatively and quantitatively demonstrated that the GSSC framework can produce truly encouraging performance with better generalization and robustness than other leading competitors.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 58 of its 84 references that are in the map. First posted on arXiv 9 Sep 2024.

Builds on

58 of 84 references.

References · 58

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2017Inductive Representation Learning on Large Graphscited by 6,284
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  6. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  7. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  8. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  9. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  10. 2019Simplifying Graph Convolutional Networkscited by 1,470
  11. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  12. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
38 more
  1. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
  2. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  3. 2020A Generalization of Transformer Networks to Graphscited by 653
  4. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
  5. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
  6. 2019Diffusion Improves Graph Learningcited by 414
  7. 2018Adversarial Attacks on Neural Networks for Graph Datacited by 395
  8. 2018Deep Learning on Graphs: A Surveycited by 393
  9. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 302
  10. 2020Towards Deeper Graph Neural Networkscited by 296
  11. 2021Graph Self-Supervised Learning: A Surveycited by 289
  12. 2020SIGN: Scalable Inception Graph Neural Networkscited by 283
  13. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  14. 2019Learning Discrete Structures for Graph Neural Networkscited by 228
  15. 2020Data Augmentation for Graph Neural Networkscited by 200
  16. 2022Finding Global Homophily in Graph Neural Networks When Meeting Heterophilycited by 179
  17. 2022SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentationcited by 171
  18. 2023Graph Inductive Biases in Transformers without Message Passingcited by 150
  19. 2021GraphiT: Encoding Graph Structure in Transformerscited by 146
  20. 2022NAGphormer: A Tokenized Graph Transformer for Node Classification in Large Graphscited by 137
  21. 2021Self-supervised Learning on Graphs: Contrastive, Generative,or Predictivecited by 136
  22. 2023SGFormer: Simplifying and Empowering Transformers for Large-Graph Representationscited by 131
  23. 2021Self-supervised Graph-level Representation Learning with Local and Global Structurecited by 125
  24. 2021SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networkscited by 108
  25. 2019Graph Adversarial Training: Dynamically Regularizing Based on Graph Structurecited by 93
  26. 2022A Generalization of ViT/MLP-Mixer to Graphscited by 93
  27. 2020Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learningcited by 90
  28. 2019Learning Graph Embedding with Adversarial Training Methodscited by 84
  29. 2021OOD-GNN: Out-of-Distribution Generalized Graph Neural Networkcited by 78
  30. 2021Graph-MLP: Node Classification without Message Passing in Graphcited by 73
  31. 2021A Survey on Graph Structure Learning: Progress and Opportunitiescited by 70
  32. 2019Graph-Revised Convolutional Networkcited by 58
  33. 2021Co-learning: Learning from Noisy Labels with Self-supervisioncited by 50
  34. 2021GraphMixup: Improving Class-Imbalanced Node Classification on Graphs by Self-supervised Context Predictioncited by 36
  35. 2019Adversarial Attack Generation Empowered by Min-Max Optimizationcited by 32
  36. 2023SGFormer: Semantic Graph Transformer for Point Cloud-based 3D Scene Graph Generationcited by 27
  37. 2023Quantifying the Knowledge in GNNs for Reliable Distillation into MLPscited by 25
  38. 2023Extracting Low-/High- Frequency Knowledge from Graph Neural Networks and Injecting it into MLPs: An Effective GNN-to-MLP Distillation Frameworkcited by 24

50 most cited of 58 All 58 (JSON)

Built on by

Not cited yet.