mlsanityData to 7 Oct 2026

Analyzing Data-Centric Properties for Graph Contrastive Learning

arXiv 2208.02810 ↗First posted 4 Aug 2022cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Recent analyses of self-supervised learning (SSL) find the following data-centric properties to be critical for learning good representations: invariance to task-irrelevant semantics, separability of classes in some latent space, and recoverability of labels from augmented samples. However, given their discrete, non-Euclidean nature, graph datasets and graph SSL methods are unlikely to satisfy these properties. This raises the question: how do graph SSL methods, such as contrastive learning (CL), work well? To systematically probe this question, we perform a generalization analysis for CL when using generic graph augmentations (GGAs), with a focus on data-centric properties. Our analysis yields formal insights into the limitations of GGAs and the necessity of task-relevant augmentations. As we empirically show, GGAs do not induce task-relevant invariances on common benchmark datasets, leading to only marginal gains over naive, untrained baselines. Our theory motivates a synthetic data generation process that enables control over task-relevant information and boasts pre-defined optimal augmentations. This flexible benchmark helps us identify yet unrecognized limitations in advanced augmentation techniques (e.g., automated methods). Overall, our work rigorously contextualizes, both empirically and theoretically, the effects of data-centric properties on augmentation strategies and learning paradigms for graph SSL.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 58 of its 69 references that are in the map. First posted on arXiv 4 Aug 2022.

Builds on

58 of 69 references.

References · 58

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2018How Powerful are Graph Neural Networks?cited by 4,355
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  10. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  11. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  12. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
38 more
  1. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  2. 2016Variational Graph Auto-Encoderscited by 1,373
  3. 2019Contrastive Multiview Codingcited by 1,297
  4. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  5. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  6. 2018Deep Graph Infomaxcited by 1,101
  7. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  8. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  9. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  10. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
  11. 2019Adversarial Training for Free!cited by 638
  12. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  13. 2019On Variational Bounds of Mutual Informationcited by 550
  14. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  15. 2020Graph Contrastive Learning with Adaptive Augmentationcited by 519
  16. 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 511
  17. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 461
  18. 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
  19. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
  20. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 311
  21. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
  22. 2021Graph Self-Supervised Learning: A Surveycited by 289
  23. 2021Graph Contrastive Learning Automatedcited by 287
  24. 2021Understanding self-supervised Learning Dynamics without Contrastive Pairscited by 254
  25. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  26. 2020How Well Do Self-Supervised Models Transfer?cited by 213
  27. 2020Data Augmentation for Graph Neural Networkscited by 200
  28. 2021Adversarial Graph Augmentation to Improve Graph Contrastive Learningcited by 200
  29. 2021Large-Scale Representation Learning on Graphs via Bootstrappingcited by 184
  30. 2020Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Datacited by 179
  31. 2022SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentationcited by 171
  32. 2020Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biasescited by 143
  33. 2021Self-supervised Graph-level Representation Learning with Local and Global Structurecited by 125
  34. 2021Self-supervised Learning is More Robust to Dataset Imbalancecited by 122
  35. 2020When Does Self-Supervision Help Graph Convolutional Networks?cited by 121
  36. 2020From Local Structures to Size Generalization in Graph Neural Networkscited by 119
  37. 2021An Empirical Study of Graph Contrastive Learningcited by 113
  38. 2021Towards the Generalization of Contrastive Self-Supervised Learningcited by 99

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 6; 2024: 0; 2025: 1.

7 papers.

  1. 2023Simple and Asymmetric Graph Contrastive Learning without Augmentationscited by 37
  2. 2023Adversarial Training for Graph Neural Networks: Pitfalls, Solutions, and New Directionscited by 28
  3. 2023A Closer Look at Model Adaptation using Feature Distortion and Simplicity Biascited by 20
  4. 2023Data-Centric Learning from Unlabeled Graphs with Diffusion Modelcited by 19
  5. 2023Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networkscited by 14
  6. 2023Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive Learningcited by 6
  7. 2025Graph Contrastive Learning versus Untrained Baselines: The Role of Dataset Sizecited by 0