# Eliciting Structural and Semantic Global Knowledge in Unsupervised Graph Contrastive Learning

arXiv: https://arxiv.org/abs/2202.08480 · HTML: https://mlsanity.com/paper/2202.08480/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 53 references that are in the map.
- First posted on arXiv 17 Feb 2022.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Predict then Propagate: Graph Neural Networks meet Personalized PageRank](https://mlsanity.com/paper/1810.05997/) (2018)
- [Representation Learning on Graphs with Jumping Knowledge Networks](https://mlsanity.com/paper/1806.03536/) (2018)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [On the Bottleneck of Graph Neural Networks and its Practical Implications](https://mlsanity.com/paper/2006.05205/) (2020)
- [Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View](https://mlsanity.com/paper/1909.03211/) (2019)
- [Deep Graph Contrastive Representation Learning](https://mlsanity.com/paper/2006.04131/) (2020)
- [Graph Contrastive Learning with Adaptive Augmentation](https://mlsanity.com/paper/2010.14945/) (2020)
- [GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training](https://mlsanity.com/paper/2006.09963/) (2020)
- [Debiased Contrastive Learning](https://mlsanity.com/paper/2007.00224/) (2020)
- [Graph Representation Learning via Graphical Mutual Information Maximization](https://mlsanity.com/paper/2002.01169/) (2020)
- [Graph Random Neural Network for Semi-Supervised Learning on Graphs](https://mlsanity.com/paper/2005.11079/) (2020)
- [Data Augmentation for Deep Graph Learning: A Survey](https://mlsanity.com/paper/2202.08235/) (2022)
- [Self-supervised Graph-level Representation Learning with Local and Global Structure](https://mlsanity.com/paper/2106.04113/) (2021)
- [Symmetric Graph Convolutional Autoencoder for Unsupervised Graph Representation Learning](https://mlsanity.com/paper/1908.02441/) (2019)

## Built on by (most cited first, 20 of 20)

- [GOOD-D: On Unsupervised Graph Out-Of-Distribution Detection](https://mlsanity.com/paper/2211.04208/) (2022)
- [Learning Strong Graph Neural Networks with Weak Information](https://mlsanity.com/paper/2305.18457/) (2023)
- [Transductive Linear Probing: A Novel Framework for Few-Shot Node Classification](https://mlsanity.com/paper/2212.05606/) (2022)
- [Toward Robust Graph Semi-Supervised Learning against Extreme Data Scarcity](https://mlsanity.com/paper/2208.12422/) (2022)
- [A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective](https://mlsanity.com/paper/2403.16137/) (2024)
- [HyperFormer: Learning Expressive Sparse Feature Representations via Hypergraph Transformer](https://mlsanity.com/paper/2305.17386/) (2023)
- [Unveiling Global Interactive Patterns across Graphs: Towards Interpretable Graph Neural Networks](https://mlsanity.com/paper/2407.01979/) (2024)
- [Graph Contrastive Learning via Cluster-refined Negative Sampling for Semi-supervised Text Classification](https://mlsanity.com/paper/2410.18130/) (2024)
- [Relational Prompt-based Pre-trained Language Models for Social Event Detection](https://mlsanity.com/paper/2404.08263/) (2024)
- [TAGA: Text-Attributed Graph Self-Supervised Learning by Synergizing Graph and Text Mutual Transformations](https://mlsanity.com/paper/2405.16800/) (2024)
- [From GNNs to Trees: Multi-Granular Interpretability for Graph Neural Networks](https://mlsanity.com/paper/2505.00364/) (2025)
- [Unsupervised Graph Clustering with Deep Structural Entropy](https://mlsanity.com/paper/2505.14040/) (2025)
- [HGOE: Hybrid External and Internal Graph Outlier Exposure for Graph Out-of-Distribution Detection](https://mlsanity.com/paper/2407.21742/) (2024)
- [RCoCo: Contrastive Collective Link Prediction across Multiplex Network in Riemannian Space](https://mlsanity.com/paper/2403.01864/) (2024)
- [CueGCL: Cluster-aware Personalized Self-Training for Unsupervised Graph Contrastive Learning](https://mlsanity.com/paper/2311.11073/) (2023)
- [Predict, Cluster, Refine: A Joint Embedding Predictive Self-Supervised Framework for Graph Representation Learning](https://mlsanity.com/paper/2502.01684/) (2025)
- [Global graph features unveiled by unsupervised geometric deep learning](https://mlsanity.com/paper/2503.05560/) (2025)
- [CoATA: Effective Co-Augmentation of Topology and Attribute for Graph Neural Networks](https://mlsanity.com/paper/2506.22299/) (2025)
- [Reconciling Attribute and Structural Anomalies for Improved Graph Anomaly Detection](https://mlsanity.com/paper/2506.23469/) (2025)
- [Explainable Graph Representation Learning via Graph Pattern Analysis](https://mlsanity.com/paper/2512.04530/) (2025)

## Abstract (arXiv)

Graph Contrastive Learning (GCL) has recently drawn much research interest for learning generalizable node representations in a self-supervised manner. In general, the contrastive learning process in GCL is performed on top of the representations learned by a graph neural network (GNN) backbone, which transforms and propagates the node contextual information based on its local neighborhoods. However, nodes sharing similar characteristics may not always be geographically close, which poses a great challenge for unsupervised GCL efforts due to their inherent limitations in capturing such global graph knowledge. In this work, we address their inherent limitations by proposing a simple yet effective framework -- Simple Neural Networks with Structural and Semantic Contrastive Learning (S3-CL). Notably, by virtue of the proposed structural and semantic contrastive learning algorithms, even a simple neural network can learn expressive node representations that preserve valuable global structural and semantic patterns. Our experiments demonstrate that the node representations learned by S3-CL achieve superior performance on different downstream tasks compared with the state-of-the-art unsupervised GCL methods. Implementation and more experimental details are publicly available at urlhttps://github.com/kaize0409/S-3-CL.