mlsanityData to 7 Oct 2026

Graph Self-Supervised Learning: A Survey

arXiv 2103.00111 ↗First posted 27 Feb 2021cs.LGcited by 289

Not on the map. Why?

Abstract (arXiv)

Deep learning on graphs has attracted significant interests recently. However, most of the works have focused on (semi-) supervised learning, resulting in shortcomings including heavy label reliance, poor generalization, and weak robustness. To address these issues, self-supervised learning (SSL), which extracts informative knowledge through well-designed pretext tasks without relying on manual labels, has become a promising and trending learning paradigm for graph data. Different from SSL on other domains like computer vision and natural language processing, SSL on graphs has an exclusive background, design ideas, and taxonomies. Under the umbrella of graph self-supervised learning, we present a timely and comprehensive review of the existing approaches which employ SSL techniques for graph data. We construct a unified framework that mathematically formalizes the paradigm of graph SSL. According to the objectives of pretext tasks, we divide these approaches into four categories: generation-based, auxiliary property-based, contrast-based, and hybrid approaches. We further describe the applications of graph SSL across various research fields and summarize the commonly used datasets, evaluation benchmark, performance comparison and open-source codes of graph SSL. Finally, we discuss the remaining challenges and potential future directions in this research field.

Key facts. Cited by 289 papers in the map (data through 7 Oct 2026). Builds on 103 of its 184 references that are in the map. First posted on arXiv 27 Feb 2021.

Builds on

103 of 184 references.

References · 103

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2017Graph Attention Networkscited by 8,854
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2017Inductive Representation Learning on Large Graphscited by 6,284
  9. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  10. 2018How Powerful are Graph Neural Networks?cited by 4,355
  11. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  12. 2013Auto-Encoding Variational Bayescited by 3,980
38 more
  1. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  2. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  3. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  4. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  5. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  6. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  7. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  8. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  9. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  10. 2016Variational Graph Auto-Encoderscited by 1,373
  11. 2019Contrastive Multiview Codingcited by 1,297
  12. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  13. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  14. 2016Colorful Image Colorizationcited by 1,113
  15. 2018Deep Graph Infomaxcited by 1,101
  16. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
  17. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  18. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  19. 2019Graph WaveNet for Deep Spatial-Temporal Graph Modelingcited by 820
  20. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 786
  21. 2020Self-supervised Learning: Generative or Contrastivecited by 693
  22. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  23. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 656
  24. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
  25. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  26. 2020Contrastive Learning with Hard Negative Samplescited by 619
  27. 2015LINE: Large-scale Information Network Embeddingcited by 582
  28. 2020A Survey on Contrastive Self-supervised Learningcited by 553
  29. 2020Deep Graph Contrastive Representation Learningcited by 535
  30. 2020Graph Contrastive Learning with Adaptive Augmentationcited by 519
  31. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 461
  32. 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
  33. 2019Diffusion Improves Graph Learningcited by 414
  34. 2019On Mutual Information Maximization for Representation Learningcited by 315
  35. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 302
  36. 2020Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networkscited by 253
  37. 2020Graph Representation Learning via Graphical Mutual Information Maximizationcited by 217
  38. 2019Contrastive Learning of Structured World Modelscited by 206

50 most cited of 103 All 103 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 24; 2022: 60; 2023: 66; 2024: 84; 2025: 30; 2026: 25.

289 papers.

Sort:

By area:

  1. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  2. 2021Graph Contrastive Learning Automatedcited by 287
  3. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  4. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 223
  5. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  6. 2021Adversarial Graph Augmentation to Improve Graph Contrastive Learningcited by 200
  7. 2023A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detectioncited by 145
  8. 2021Self-supervised Learning on Graphs: Contrastive, Generative,or Predictivecited by 136
  9. 2022Data Augmentation for Deep Graph Learning: A Surveycited by 132
  10. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  11. 2021An Empirical Study of Graph Contrastive Learningcited by 113
  12. 2022Towards Unsupervised Deep Graph Structure Learningcited by 112
38 more
  1. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 101
  2. 2022Universal Prompt Tuning for Graph Neural Networkscited by 100
  3. 2022Out-Of-Distribution Generalization on Graphs: A Surveycited by 97
  4. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  5. 2022Trustworthy Graph Neural Networks: Aspects, Methods and Trendscited by 88
  6. 2022Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learningcited by 86
  7. 2022Self-Supervised Learning for Videos: A Surveycited by 85
  8. 2024All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretrainingcited by 84
  9. 2021Self-supervised learning methods and applications in medical imaging analysis: A surveycited by 76
  10. 2024OpenGraph: Towards Open Graph Foundation Modelscited by 75
  11. 2022Multivariate Time Series Forecasting with Dynamic Graph Neural ODEscited by 72
  12. 2023Structure-free Graph Condensation: From Large-scale Graphs to Condensed Graph-free Datacited by 71
  13. 2021Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detectioncited by 67
  14. 2022Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matchingcited by 67
  15. 2022Empowering Graph Representation Learning with Test-Time Graph Transformationcited by 66
  16. 2021Anomaly Detection in Dynamic Graphs via Transformercited by 57
  17. 2022Beyond Smoothing: Unsupervised Graph Representation Learning with Edge Heterophily Discriminatingcited by 57
  18. 2024GOFA: A Generative One-For-All Model for Joint Graph Language Modelingcited by 56
  19. 2022Graph Anomaly Detection via Multi-Scale Contrastive Learning Networks with Augmented Viewcited by 54
  20. 2024AnyGraph: Graph Foundation Model in the Wildcited by 51
  21. 2021Adapting Membership Inference Attacks to GNN for Graph Classification: Approaches and Implicationscited by 50
  22. 2022Heterogeneous Graph Masked Autoencoderscited by 47
  23. 2022GOOD-D: On Unsupervised Graph Out-Of-Distribution Detectioncited by 47
  24. 2024Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insightscited by 44
  25. 2023Dink-Net: Neural Clustering on Large Graphscited by 38
  26. 2021Prototypical Graph Contrastive Learningcited by 37
  27. 2021When Do Contrastive Learning Signals Help Spatio-Temporal Graph Forecasting?cited by 36
  28. 2024On the Scalability of GNNs for Molecular Graphscited by 36
  29. 2021Graph Communal Contrastive Learningcited by 34
  30. 2022Jointly Contrastive Representation Learning on Road Network and Trajectorycited by 34
  31. 2023A Group Symmetric Stochastic Differential Equation Model for Molecule Multi-modal Pretrainingcited by 32
  32. 2023HetGPT: Harnessing the Power of Prompt Tuning in Pre-Trained Heterogeneous Graph Neural Networkscited by 32
  33. 2022SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Drivingcited by 31
  34. 2023Graph Positional and Structural Encodercited by 31
  35. 2024Federated Graph Semantic and Structural Learningcited by 30
  36. 2022Uncovering the Structural Fairness in Graph Contrastive Learningcited by 29
  37. 2022Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentationscited by 28
  38. 2022A Survey of Deep Graph Clustering: Taxonomy, Challenge, Application, and Open Resourcecited by 26

50 most cited of 289 All 289 (JSON)