mlsanityData to 7 Oct 2026

COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning

arXiv 2206.04726 ↗First posted 9 Jun 2022cs.LG cs.AIcited by 38

Not on the map. Why?

Abstract (arXiv)

Graph contrastive learning (GCL) improves graph representation learning, leading to SOTA on various downstream tasks. The graph augmentation step is a vital but scarcely studied step of GCL. In this paper, we show that the node embedding obtained via the graph augmentations is highly biased, somewhat limiting contrastive models from learning discriminative features for downstream tasks. Thus, instead of investigating graph augmentation in the input space, we alternatively propose to perform augmentations on the hidden features (feature augmentation). Inspired by so-called matrix sketching, we propose COSTA, a novel COvariance-preServing feaTure space Augmentation framework for GCL, which generates augmented features by maintaining a "good sketch" of original features. To highlight the superiority of feature augmentation with COSTA, we investigate a single-view setting (in addition to multi-view one) which conserves memory and computations. We show that the feature augmentation with COSTA achieves comparable/better results than graph augmentation based models.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 37 of its 52 references that are in the map. First posted on arXiv 9 Jun 2022.

Builds on

37 of 52 references.

References · 37

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  6. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  7. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  8. 2016Variational Graph Auto-Encoderscited by 1,373
  9. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  10. 2018Deep Graph Infomaxcited by 1,101
  11. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  12. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
25 more
  1. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  2. 2015Image-based Recommendations on Styles and Substitutescited by 702
  3. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
  4. 2020Deep Graph Contrastive Representation Learningcited by 535
  5. 2020Graph Contrastive Learning with Adaptive Augmentationcited by 519
  6. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  7. 2020Self-supervised Graph Learning for Recommendationcited by 305
  8. 2020Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networkscited by 253
  9. 2020Graph Representation Learning via Graphical Mutual Information Maximizationcited by 217
  10. 2021Adversarial Graph Augmentation to Improve Graph Contrastive Learningcited by 200
  11. 2021Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendationcited by 173
  12. 2017Dataset Augmentation in Feature Spacecited by 124
  13. 2019Implicit Semantic Data Augmentation for Deep Networkscited by 106
  14. 2021Graph Barlow Twins: A self-supervised representation learning framework for graphscited by 85
  15. 2021Graph-based Semi-supervised Learning: A Comprehensive Reviewcited by 79
  16. 2021Modeling Scale-free Graphs with Hyperbolic Geometry for Knowledge-aware Recommendationcited by 38
  17. 2022HRCF: Enhancing Collaborative Filtering via Hyperbolic Geometric Regularizationcited by 34
  18. 2020Power Normalizations in Fine-grained Image, Few-shot Image and Graph Classificationcited by 27
  19. 2020GraphCL: Contrastive Self-Supervised Learning of Graph Representationscited by 26
  20. 2022Contrastive Laplacian Eigenmapscited by 24
  21. 2021FeatureNorm: L2 Feature Normalization for Dynamic Graph Embeddingcited by 19
  22. 2019Fisher-Bures Adversary Graph Convolutional Networkscited by 18
  23. 2021Attentive Knowledge-aware Graph Convolutional Networks with Collaborative Guidance for Personalized Recommendationcited by 12
  24. 2022Graph-adaptive Rectified Linear Unit for Graph Neural Networkscited by 12
  25. 2021REFINE: Random RangE FInder for Network Embeddingcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 11; 2024: 12; 2025: 6; 2026: 1.

38 papers.

Sort:

By area:

  1. 2022Spectral Feature Augmentation for Graph Contrastive Learning and Beyondcited by 49
  2. 20233Mformer: Multi-order Multi-mode Transformer for Skeletal Action Recognitioncited by 40
  3. 2023GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learningcited by 32
  4. 2023A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacycited by 24
  5. 2022Uncertainty-DTW for Time Series and Sequencescited by 21
  6. 2024GraphLoRA: Structure-Aware Contrastive Low-Rank Adaptation for Cross-Graph Transfer Learningcited by 21
  7. 2023HomoGCL: Rethinking Homophily in Graph Contrastive Learningcited by 18
  8. 2022Graph Soft-Contrastive Learning via Neighborhood Rankingcited by 16
  9. 2022kHGCN: Tree-likeness Modeling via Continuous and Discrete Curvature Learningcited by 14
  10. 2022Wiener Graph Deconvolutional Network Improves Graph Self-Supervised Learningcited by 13
  11. 2023Traffic Forecasting on New Roads Using Spatial Contrastive Pre-Training (SCPT)cited by 11
  12. 2024Masked Graph Autoencoder with Non-discrete Bandwidthscited by 11
26 more
  1. 2024Noise-Resilient Unsupervised Graph Representation Learning via Multi-Hop Feature Quality Estimationcited by 11
  2. 2023Rethinking and Simplifying Bootstrapped Graph Latentscited by 10
  3. 2023Understanding and Mitigating Hyperbolic Dimensional Collapse in Graph Contrastive Learningcited by 9
  4. 2024A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspectivecited by 9
  5. 2022Towards Generalizable Graph Contrastive Learning: An Information Theory Perspectivecited by 8
  6. 2023MARIO: Model Agnostic Recipe for Improving OOD Generalization of Graph Contrastive Learningcited by 7
  7. 2022Graph Component Contrastive Learning for Concept Relatedness Estimationcited by 6
  8. 2024Bootstrap Latents of Nodes and Neighbors for Graph Self-Supervised Learningcited by 6
  9. 2024Negative-Free Self-Supervised Gaussian Embedding of Graphscited by 5
  10. 2023Spectral-Aware Augmentation for Enhanced Graph Representation Learningcited by 4
  11. 2024From Overfitting to Robustness: Quantity, Quality, and Variety Oriented Negative Sample Selection in Graph Contrastive Learningcited by 4
  12. 2024Overcoming Pitfalls in Graph Contrastive Learning Evaluation: Toward Comprehensive Benchmarkscited by 3
  13. 2025InfoNCE is a Free Lunch for Semantically guided Graph Contrastive Learningcited by 3
  14. 2022Momentum Contrastive Pre-training for Question Answeringcited by 2
  15. 2024Negative as Positive: Enhancing Out-of-distribution Generalization for Graph Contrastive Learningcited by 2
  16. 2023FedHGN: A Federated Framework for Heterogeneous Graph Neural Networkscited by 1
  17. 2024Hi-GMAE: Hierarchical Graph Masked Autoencoderscited by 1
  18. 2024Topology Reorganized Graph Contrastive Learning with Mitigating Semantic Driftcited by 1
  19. 2025AS-GCL: Asymmetric Spectral Augmentation on Graph Contrastive Learningcited by 1
  20. 2025Graph Self-Supervised Learning with Learnable Structural and Positional Encodingscited by 1
  21. 2025A Binary Classification Social Network Dataset for Graph Machine Learningcited by 1
  22. 2023ID-MixGCL: Identity Mixup for Graph Contrastive Learningcited by 0
  23. 2024Multiplex Graph Contrastive Learning with Soft Negativescited by 0
  24. 2025Directed Link Prediction using GNN with Local and Global Feature Fusioncited by 0
  25. 2025Attacking and Securing Community Detection: A Game-Theoretic Frameworkcited by 0
  26. 2026Adaptive Negative Scheduling for Graph Contrastive Learningcited by 0