mlsanityData to 7 Oct 2026

Self-supervised Learning on Graphs: Deep Insights and New Direction

arXiv 2006.10141 ↗First posted 17 Jun 2020cs.LG stat.MLcited by 108

Not on the map. Why?

Abstract (arXiv)

The success of deep learning notoriously requires larger amounts of costly annotated data. This has led to the development of self-supervised learning (SSL) that aims to alleviate this limitation by creating domain specific pretext tasks on unlabeled data. Simultaneously, there are increasing interests in generalizing deep learning to the graph domain in the form of graph neural networks (GNNs). GNNs can naturally utilize unlabeled nodes through the simple neighborhood aggregation that is unable to thoroughly make use of unlabeled nodes. Thus, we seek to harness SSL for GNNs to fully exploit the unlabeled data. Different from data instances in the image and text domains, nodes in graphs present unique structure information and they are inherently linked indicating not independent and identically distributed (or i.i.d.). Such complexity is a double-edged sword for SSL on graphs. On the one hand, it determines that it is challenging to adopt solutions from the image and text domains to graphs and dedicated efforts are desired. On the other hand, it provides rich information that enables us to build SSL from a variety of perspectives. Thus, in this paper, we first deepen our understandings on when, why, and which strategies of SSL work with GNNs by empirically studying numerous basic SSL pretext tasks on graphs. Inspired by deep insights from the empirical studies, we propose a new direction SelfTask to build advanced pretext tasks that are able to achieve state-of-the-art performance on various real-world datasets. The specific experimental settings to reproduce our results can be found in urlhttps://github.com/ChandlerBang/SelfTask-GNN.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 35; 2023: 17; 2024: 18; 2025: 12; 2026: 1.

108 papers.

Sort:

By area:

  1. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
  2. 2021Graph Self-Supervised Learning: A Surveycited by 289
  3. 2021Graph Contrastive Learning Automatedcited by 287
  4. 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
  5. 2023All in One: Multi-task Prompting for Graph Neural Networkscited by 147
  6. 2021Self-supervised Learning on Graphs: Contrastive, Generative,or Predictivecited by 136
  7. 2021SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networkscited by 108
  8. 2022Universal Prompt Tuning for Graph Neural Networkscited by 100
  9. 2024Position: Graph Foundation Models are Already Herecited by 84
  10. 2021DIG: A Turnkey Library for Diving into Graph Deep Learning Researchcited by 76
  11. 2021Towards Self-Explainable Graph Neural Networkcited by 75
  12. 2021Graph-MLP: Node Classification without Message Passing in Graphcited by 73
38 more
  1. 2021FairDrop: Biased Edge Dropout for Enhancing Fairness in Graph Representation Learningcited by 68
  2. 2022Towards Robust Graph Neural Networks for Noisy Graphs with Sparse Labelscited by 63
  3. 2022Improving Fairness in Graph Neural Networks via Mitigating Sensitive Attribute Leakagecited by 61
  4. 2022Graph-based Molecular Representation Learningcited by 51
  5. 2021Automated Self-Supervised Learning for Graphscited by 50
  6. 2025Multi-Domain Graph Foundation Models: Robust Knowledge Transfer via Topology Alignmentcited by 41
  7. 2022Spectral Augmentation for Self-Supervised Learning on Graphscited by 38
  8. 2021GraphMixup: Improving Class-Imbalanced Node Classification on Graphs by Self-supervised Context Predictioncited by 36
  9. 2021Tree Decomposed Graph Neural Networkcited by 35
  10. 2023SGL-PT: A Strong Graph Learner with Graph Prompt Tuningcited by 35
  11. 2021Hop-Count Based Self-Supervised Anomaly Detection on Attributed Networkscited by 31
  12. 2021Adaptive Transfer Learning on Graph Neural Networkscited by 31
  13. 2022SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Drivingcited by 31
  14. 2022MLPInit: Embarrassingly Simple GNN Training Acceleration with MLP Initializationcited by 31
  15. 2024Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Treescited by 30
  16. 2022Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentationscited by 28
  17. 2022Augmentations in Hypergraph Contrastive Learning: Fabricated and Generativecited by 28
  18. 2022MGAE: Masked Autoencoders for Self-Supervised Learning on Graphscited by 27
  19. 2022Disentangled Contrastive Learning for Social Recommendationcited by 27
  20. 2024Towards Graph Contrastive Learning: A Survey and Beyondcited by 26
  21. 2021Curvature Graph Neural Networkcited by 24
  22. 2022Exploring Edge Disentanglement for Node Classificationcited by 23
  23. 2022Graph Anomaly Detection with Graph Neural Networks: Current Status and Challengescited by 23
  24. 2023ADA-GAD: Anomaly-Denoised Autoencoders for Graph Anomaly Detectioncited by 23
  25. 2022Modularity-Aware Graph Autoencoders for Joint Community Detection and Link Predictioncited by 22
  26. 2022Self-Supervised Representation Learning via Latent Graph Predictioncited by 22
  27. 2023A Survey on Privacy in Graph Neural Networks: Attacks, Preservation, and Applicationscited by 22
  28. 2023Graph-level Representation Learning with Joint-Embedding Predictive Architecturescited by 21
  29. 2022Self-Supervised Graph Structure Refinement for Graph Neural Networkscited by 20
  30. 2021Semi-Supervised Graph-to-Graph Translationcited by 18
  31. 2021Jointly Attacking Graph Neural Network and its Explanationscited by 18
  32. 2024A Survey of Data-Efficient Graph Learningcited by 18
  33. 2025Edge Prompt Tuning for Graph Neural Networkscited by 18
  34. 2021Graph Neural Networks: Methods, Applications, and Opportunitiescited by 16
  35. 2023Towards Data-centric Graph Machine Learning: Review and Outlookcited by 15
  36. 2022Graph Contrastive Learning with Personalized Augmentationcited by 14
  37. 2022Multi-task Joint Strategies of Self-supervised Representation Learning on Biomedical Networks for Drug Discoverycited by 13
  38. 2022Link Prediction with Non-Contrastive Learningcited by 11

50 most cited of 108 All 108 (JSON)