# Augmentation-Free Self-Supervised Learning on Graphs

arXiv: https://arxiv.org/abs/2112.02472 · HTML: https://mlsanity.com/paper/2112.02472/

## Key facts

- Cited by 94 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 47 references that are in the map.
- First posted on arXiv 5 Dec 2021.

## Builds on (most cited first, 30 of 41)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [Deep Graph Contrastive Representation Learning](https://mlsanity.com/paper/2006.04131/) (2020)
- [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://mlsanity.com/paper/1902.09229/) (2019)
- [Graph Contrastive Learning with Adaptive Augmentation](https://mlsanity.com/paper/2010.14945/) (2020)
- [GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training](https://mlsanity.com/paper/2006.09963/) (2020)
- [InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization](https://mlsanity.com/paper/1908.01000/) (2019)
- [Graph Contrastive Learning Automated](https://mlsanity.com/paper/2106.07594/) (2021)
- [Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks](https://mlsanity.com/paper/2007.02901/) (2020)
- [Graph Representation Learning via Graphical Mutual Information Maximization](https://mlsanity.com/paper/2002.01169/) (2020)
- [Data Augmentation for Graph Neural Networks](https://mlsanity.com/paper/2006.06830/) (2020)

## Built on by (most cited first, 30 of 94)

- [Data Augmentation for Deep Graph Learning: A Survey](https://mlsanity.com/paper/2202.08235/) (2022)
- [What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoders](https://mlsanity.com/paper/2205.10053/) (2022)
- [OpenGraph: Towards Open Graph Foundation Models](https://mlsanity.com/paper/2403.01121/) (2024)
- [Graph Data Augmentation for Graph Machine Learning: A Survey](https://mlsanity.com/paper/2202.08871/) (2022)
- [Hard Sample Aware Network for Contrastive Deep Graph Clustering](https://mlsanity.com/paper/2212.08665/) (2022)
- [Cluster-guided Contrastive Graph Clustering Network](https://mlsanity.com/paper/2301.01098/) (2023)
- [Simple Contrastive Graph Clustering](https://mlsanity.com/paper/2205.07865/) (2022)
- [Revisiting Graph Contrastive Learning from the Perspective of Graph Spectrum](https://mlsanity.com/paper/2210.02330/) (2022)
- [Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure](https://mlsanity.com/paper/2211.10738/) (2022)
- [Spectral Augmentation for Self-Supervised Learning on Graphs](https://mlsanity.com/paper/2210.00643/) (2022)
- [Dink-Net: Neural Clustering on Large Graphs](https://mlsanity.com/paper/2305.18405/) (2023)
- [GRENADE: Graph-Centric Language Model for Self-Supervised Representation Learning on Text-Attributed Graphs](https://mlsanity.com/paper/2310.15109/) (2023)
- [A Survey of Deep Graph Clustering: Taxonomy, Challenge, Application, and Open Resource](https://mlsanity.com/paper/2211.12875/) (2022)
- [CONVERT:Contrastive Graph Clustering with Reliable Augmentation](https://mlsanity.com/paper/2308.08963/) (2023)
- [ADA-GAD: Anomaly-Denoised Autoencoders for Graph Anomaly Detection](https://mlsanity.com/paper/2312.14535/) (2023)
- [Graph-level Representation Learning with Joint-Embedding Predictive Architectures](https://mlsanity.com/paper/2309.16014/) (2023)
- [Augmentation-Free Graph Contrastive Learning of Invariant-Discriminative Representations](https://mlsanity.com/paper/2210.08345/) (2022)
- [HomoGCL: Rethinking Homophily in Graph Contrastive Learning](https://mlsanity.com/paper/2306.09614/) (2023)
- [Revisiting Modularity Maximization for Graph Clustering: A Contrastive Learning Perspective](https://mlsanity.com/paper/2406.14288/) (2024)
- [GAugLLM: Improving Graph Contrastive Learning for Text-Attributed Graphs with Large Language Models](https://mlsanity.com/paper/2406.11945/) (2024)
- [Graph Soft-Contrastive Learning via Neighborhood Ranking](https://mlsanity.com/paper/2209.13964/) (2022)
- [Heterogeneous Graph Learning for Multi-modal Medical Data Analysis](https://mlsanity.com/paper/2211.15158/) (2022)
- [Reinforcement Graph Clustering with Unknown Cluster Number](https://mlsanity.com/paper/2308.06827/) (2023)
- [Shift-Robust Molecular Relational Learning with Causal Substructure](https://mlsanity.com/paper/2305.18451/) (2023)
- [Single-Pass Contrastive Learning Can Work for Both Homophilic and Heterophilic Graph](https://mlsanity.com/paper/2211.10890/) (2022)
- [Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks](https://mlsanity.com/paper/2311.01038/) (2023)
- [PyG-SSL: A Graph Self-Supervised Learning Toolkit](https://mlsanity.com/paper/2412.21151/) (2024)
- [Wiener Graph Deconvolutional Network Improves Graph Self-Supervised Learning](https://mlsanity.com/paper/2206.12933/) (2022)
- [STERLING: Synergistic Representation Learning on Bipartite Graphs](https://mlsanity.com/paper/2302.05428/) (2023)
- [Relational Self-Supervised Learning on Graphs](https://mlsanity.com/paper/2208.10493/) (2022)

## Abstract (arXiv)

Inspired by the recent success of self-supervised methods applied on images, self-supervised learning on graph structured data has seen rapid growth especially centered on augmentation-based contrastive methods. However, we argue that without carefully designed augmentation techniques, augmentations on graphs may behave arbitrarily in that the underlying semantics of graphs can drastically change. As a consequence, the performance of existing augmentation-based methods is highly dependent on the choice of augmentation scheme, i.e., hyperparameters associated with augmentations. In this paper, we propose a novel augmentation-free self-supervised learning framework for graphs, named AFGRL. Specifically, we generate an alternative view of a graph by discovering nodes that share the local structural information and the global semantics with the graph. Extensive experiments towards various node-level tasks, i.e., node classification, clustering, and similarity search on various real-world datasets demonstrate the superiority of AFGRL. The source code for AFGRL is available at https://github.com/Namkyeong/AFGRL.