# Augmentations in Graph Contrastive Learning: Current Methodological Flaws & Towards Better Practices

arXiv: https://arxiv.org/abs/2111.03220 · HTML: https://mlsanity.com/paper/2111.03220/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 104 references that are in the map.
- First posted on arXiv 5 Nov 2021.

## Builds on (most cited first, 30 of 69)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [TUDataset: A collection of benchmark datasets for learning with graphs](https://mlsanity.com/paper/2007.08663/) (2020)
- [Benchmarking Graph Neural Networks](https://mlsanity.com/paper/2003.00982/) (2020)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)

## Built on by (most cited first, 25 of 25)

- [GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner](https://mlsanity.com/paper/2304.04779/) (2023)
- [Graph Data Augmentation for Graph Machine Learning: A Survey](https://mlsanity.com/paper/2202.08871/) (2022)
- [Simple Contrastive Graph Clustering](https://mlsanity.com/paper/2205.07865/) (2022)
- [Mechanistic Mode Connectivity](https://mlsanity.com/paper/2211.08422/) (2022)
- [Augmentations in Hypergraph Contrastive Learning: Fabricated and Generative](https://mlsanity.com/paper/2210.03801/) (2022)
- [Automated Spatio-Temporal Graph Contrastive Learning](https://mlsanity.com/paper/2305.03920/) (2023)
- [Automated Data Augmentations for Graph Classification](https://mlsanity.com/paper/2202.13248/) (2022)
- [HomoGCL: Rethinking Homophily in Graph Contrastive Learning](https://mlsanity.com/paper/2306.09614/) (2023)
- [Augmentation-Free Graph Contrastive Learning with Performance Guarantee](https://mlsanity.com/paper/2204.04874/) (2022)
- [Single-Pass Contrastive Learning Can Work for Both Homophilic and Heterophilic Graph](https://mlsanity.com/paper/2211.10890/) (2022)
- [ES-GNN: Generalizing Graph Neural Networks Beyond Homophily with Edge Splitting](https://mlsanity.com/paper/2205.13700/) (2022)
- [Robust Graph Structure Learning under Heterophily](https://mlsanity.com/paper/2403.03659/) (2024)
- [Decoupled Self-supervised Learning for Non-Homophilous Graphs](https://mlsanity.com/paper/2206.03601/) (2022)
- [Data Augmentation on Graphs: A Technical Survey](https://mlsanity.com/paper/2212.09970/) (2022)
- [Revisiting, Benchmarking and Understanding Unsupervised Graph Domain Adaptation](https://mlsanity.com/paper/2407.11052/) (2024)
- [Analyzing Data-Centric Properties for Graph Contrastive Learning](https://mlsanity.com/paper/2208.02810/) (2022)
- [Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive Learning](https://mlsanity.com/paper/2311.02687/) (2023)
- [Enhancing Size Generalization in Graph Neural Networks through Disentangled Representation Learning](https://mlsanity.com/paper/2406.04601/) (2024)
- [Graph Contrastive Learning with Cohesive Subgraph Awareness](https://mlsanity.com/paper/2401.17580/) (2024)
- [Improving Molecular Pretraining with Complementary Featurizations](https://mlsanity.com/paper/2209.15101/) (2022)
- [Spectral-Aware Augmentation for Enhanced Graph Representation Learning](https://mlsanity.com/paper/2310.13845/) (2023)
- [UGMAE: A Unified Framework for Graph Masked Autoencoders](https://mlsanity.com/paper/2402.08023/) (2024)
- [Node Embeddings via Neighbor Embeddings](https://mlsanity.com/paper/2503.23822/) (2025)
- [Graph Contrastive Learning versus Untrained Baselines: The Role of Dataset Size](https://mlsanity.com/paper/2509.01541/) (2025)
- [Complementary Supervised and Self-Supervised Representations for Out-of-Distribution Graph Learning](https://mlsanity.com/paper/2610.07628/) (2026)

## Abstract (arXiv)

Unsupervised graph representation learning is critical to a wide range of applications where labels may be scarce or expensive to procure. Contrastive learning (CL) is an increasingly popular paradigm for such settings and the state-of-the-art in unsupervised visual representation learning. Recent work attributes the success of visual CL to use of task-relevant augmentations and large, diverse datasets. Interestingly, graph CL frameworks report strong performance despite using orders of magnitude smaller datasets and employing domain-agnostic graph augmentations (DAGAs). Motivated by this discrepancy, we probe the quality of representations learnt by popular graph CL frameworks using DAGAs. We find that DAGAs can destroy task-relevant information and harm the model's ability to learn discriminative representations. On small benchmark datasets, we show the inductive bias of graph neural networks can significantly compensate for this weak discriminability. Based on our findings, we propose several sanity checks that enable practitioners to quickly assess the quality of their model's learned representations. We further propose a broad strategy for designing task-aware augmentations that are amenable to graph CL and demonstrate its efficacy on two large-scale, complex graph applications. For example, in graph-based document classification, we show task-aware augmentations improve accuracy up to 20%.