# Self-supervised Graph Neural Networks without explicit negative sampling

arXiv: https://arxiv.org/abs/2103.14958 · HTML: https://mlsanity.com/paper/2103.14958/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 34 references that are in the map.
- First posted on arXiv 27 Mar 2021.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Revisiting Semi-Supervised Learning with Graph Embeddings](https://mlsanity.com/paper/1603.08861/) (2016)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [CURL: Contrastive Unsupervised Representations for Reinforcement Learning](https://mlsanity.com/paper/2004.04136/) (2020)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [GraphSAINT: Graph Sampling Based Inductive Learning Method](https://mlsanity.com/paper/1907.04931/) (2019)
- [FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling](https://mlsanity.com/paper/1801.10247/) (2018)
- [LINE: Large-scale Information Network Embedding](https://mlsanity.com/paper/1503.03578/) (2015)
- [Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks](https://mlsanity.com/paper/1905.07953/) (2019)
- [GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training](https://mlsanity.com/paper/2006.09963/) (2020)
- [Diffusion Improves Graph Learning](https://mlsanity.com/paper/1911.05485/) (2019)
- [GPT-GNN: Generative Pre-Training of Graph Neural Networks](https://mlsanity.com/paper/2006.15437/) (2020)
- [Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks](https://mlsanity.com/paper/1911.07323/) (2019)
- [Multi-Stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labels](https://mlsanity.com/paper/1902.11038/) (2019)
- [BYOL works even without batch statistics](https://mlsanity.com/paper/2010.10241/) (2020)
- [Understanding Self-supervised Learning with Dual Deep Networks](https://mlsanity.com/paper/2010.00578/) (2020)
- [Minimal Variance Sampling with Provable Guarantees for Fast Training of Graph Neural Networks](https://mlsanity.com/paper/2006.13866/) (2020)
- [A simple yet effective baseline for non-attributed graph classification](https://mlsanity.com/paper/1811.03508/) (2018)

## Built on by (most cited first, 25 of 25)

- [GraphMAE: Self-Supervised Masked Graph Autoencoders](https://mlsanity.com/paper/2205.10803/) (2022)
- [Self-supervised Learning on Graphs: Contrastive, Generative,or Predictive](https://mlsanity.com/paper/2105.07342/) (2021)
- [An Empirical Study of Graph Contrastive Learning](https://mlsanity.com/paper/2109.01116/) (2021)
- [Augmentation-Free Self-Supervised Learning on Graphs](https://mlsanity.com/paper/2112.02472/) (2021)
- [A Comprehensive Survey on Data Augmentation](https://mlsanity.com/paper/2405.09591/) (2024)
- [Augmentations in Graph Contrastive Learning: Current Methodological Flaws & Towards Better Practices](https://mlsanity.com/paper/2111.03220/) (2021)
- [Link Prediction with Non-Contrastive Learning](https://mlsanity.com/paper/2211.14394/) (2022)
- [Data Augmentation on Graphs: A Technical Survey](https://mlsanity.com/paper/2212.09970/) (2022)
- [A Teacher-Free Graph Knowledge Distillation Framework with Dual Self-Distillation](https://mlsanity.com/paper/2403.03483/) (2024)
- [Analyzing Data-Centric Properties for Graph Contrastive Learning](https://mlsanity.com/paper/2208.02810/) (2022)
- [Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive Learning](https://mlsanity.com/paper/2311.02687/) (2023)
- [CARL-G: Clustering-Accelerated Representation Learning on Graphs](https://mlsanity.com/paper/2306.06936/) (2023)
- [Features Based Adaptive Augmentation for Graph Contrastive Learning](https://mlsanity.com/paper/2207.01792/) (2022)
- [Revisiting Graph Contrastive Learning for Anomaly Detection](https://mlsanity.com/paper/2305.02496/) (2023)
- [Uncertainty in GNN Learning Evaluations: The Importance of a Consistent Benchmark for Community Detection](https://mlsanity.com/paper/2305.06026/) (2023)
- [Jointly Learnable Data Augmentations for Self-Supervised GNNs](https://mlsanity.com/paper/2108.10420/) (2021)
- [Uncertainty in GNN Learning Evaluations: A Comparison Between Measures for Quantifying Randomness in GNN Community Detection](https://mlsanity.com/paper/2312.09015/) (2023)
- [Tensor-Fused Multi-View Graph Contrastive Learning](https://mlsanity.com/paper/2410.15247/) (2024)
- [Graph Prompting for Graph Learning Models: Recent Advances and Future Directions](https://mlsanity.com/paper/2506.08326/) (2025)
- [Self-Supervised Node Representation Learning via Node-to-Neighbourhood Alignment](https://mlsanity.com/paper/2302.04626/) (2023)
- [Federated Graph Representation Learning using Self-Supervision](https://mlsanity.com/paper/2210.15120/) (2022)
- [Data-Driven Self-Supervised Graph Representation Learning](https://mlsanity.com/paper/2412.18316/) (2024)
- [Generative and Contrastive Graph Representation Learning](https://mlsanity.com/paper/2505.11776/) (2025)
- [Weak Supervision for Real World Graphs](https://mlsanity.com/paper/2506.02451/) (2025)
- [Beyond Augmentation: Leveraging Inter-Instance Relation in Self-Supervised Representation Learning](https://mlsanity.com/paper/2510.22322/) (2025)

## Abstract (arXiv)

Real world data is mostly unlabeled or only few instances are labeled. Manually labeling data is a very expensive and daunting task. This calls for unsupervised learning techniques that are powerful enough to achieve comparable results as semi-supervised/supervised techniques. Contrastive self-supervised learning has emerged as a powerful direction, in some cases outperforming supervised techniques. In this study, we propose, SelfGNN, a novel contrastive self-supervised graph neural network (GNN) without relying on explicit contrastive terms. We leverage Batch Normalization, which introduces implicit contrastive terms, without sacrificing performance. Furthermore, as data augmentation is key in contrastive learning, we introduce four feature augmentation (FA) techniques for graphs. Though graph topological augmentation (TA) is commonly used, our empirical findings show that FA perform as good as TA. Moreover, FA incurs no computational overhead, unlike TA, which often has O(N^3) time complexity, N-number of nodes. Our empirical evaluation on seven publicly available real-world data shows that, SelfGNN is powerful and leads to a performance comparable with SOTA supervised GNNs and always better than SOTA semi-supervised and unsupervised GNNs. The source code is available at https://github.com/zekarias-tilahun/SelfGNN.