mlsanityData to 14 Oct 2026

Revisiting Graph Autoencoders as Implicit Contrastive Learners

arXiv 2410.10241 ↗First posted 14 Oct 2024cs.LG cs.AI stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

Graph autoencoders (GAEs) and graph contrastive learning (GCL) are two major paradigms for self-supervised representation learning on graphs, yet they are often studied in isolation and treated as fundamentally different approaches. In this work, we revisit GAEs through the lens of contrastive learning and show that both structure-based and feature-based GAEs can be conceptualized as implicitly graph contrastive learners. This perspective reveals that many existing GAEs differ primarily in how contrastive views are constructed, rather than in their learning objectives or architectures. Building on this insight, we introduce a unified formulation that highlights contrastive view design as a central and previously less explored dimension in GAEs. In particular, we identify asymmetric contrastive views, arising from mismatches in subgraph views, as an important yet underexplored design axis in prior GAE research. We formalize this insight within a unified framework and conduct systematic experiments on representative graph learning tasks to examine its impact on performance and efficiency. Our results show that interpreting GAEs as implicit contrastive learners offers a clearer understanding of existing models and provides practical guidance for designing effective and scalable graph autoencoders.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 48 of its 60 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

48 of 60 references.

References · 48

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2017Graph Attention Networkscited by 8,923
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2017Inductive Representation Learning on Large Graphscited by 6,320
  7. 2018How Powerful are Graph Neural Networks?cited by 4,393
  8. 2013Auto-Encoding Variational Bayescited by 4,053
  9. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  10. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  11. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  12. 2016node2vec: Scalable Feature Learning for Networkscited by 1,918
36 more
  1. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
  2. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  3. 2016Variational Graph Auto-Encoderscited by 1,382
  4. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  5. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  6. 2018Deep Graph Infomaxcited by 1,106
  7. 2018Pitfalls of Graph Neural Network Evaluationcited by 1,007
  8. 2019Heterogeneous Graph Attention Networkcited by 734
  9. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 680
  10. 2020Contrastive Multi-View Representation Learning on Graphscited by 652
  11. 2020Heterogeneous Graph Transformercited by 582
  12. 2020Deep Graph Contrastive Representation Learningcited by 539
  13. 2020Graph Contrastive Learning with Adaptive Augmentationcited by 519
  14. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 448
  15. 2019On Mutual Information Maximization for Representation Learningcited by 316
  16. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 313
  17. 2021Adversarial Graph Augmentation to Improve Graph Contrastive Learningcited by 200
  18. 2021Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networkscited by 176
  19. 2020Predicting What You Already Know Helps: Provable Self-Supervised Learningcited by 152
  20. 2022Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methodscited by 149
  21. 2021From Canonical Correlation Analysis to Self-supervised Graph Neural Networkscited by 141
  22. 2021Self-supervised Learning on Graphs: Contrastive, Generative,or Predictivecited by 136
  23. 2023GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learnercited by 132
  24. 2021InfoGCL: Information-Aware Graph Contrastive Learningcited by 105
  25. 2021Towards the Generalization of Contrastive Self-Supervised Learningcited by 99
  26. 2021Augmentation-Free Self-Supervised Learning on Graphscited by 94
  27. 2022How Mask Matters: Towards Theoretical Understandings of Masked Autoencoderscited by 86
  28. 2018Adversarially Regularized Graph Autoencoder for Graph Embeddingcited by 84
  29. 2022What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoderscited by 82
  30. 2022Rethinking and Scaling Up Graph Contrastive Learning: An Extremely Efficient Approach with Group Discriminationcited by 69
  31. 2019Semi-Implicit Graph Variational Auto-Encoderscited by 54
  32. 2023InfoNCE Loss Provably Learns Cluster-Preserving Representationscited by 37
  33. 2023SeeGera: Self-supervised Semi-implicit Graph Variational Auto-encoders with Maskingcited by 23
  34. 2024Rethinking Graph Masked Autoencoders through Alignment and Uniformitycited by 22
  35. 2023GiGaMAE: Generalizable Graph Masked Autoencoder via Collaborative Latent Space Reconstructioncited by 21
  36. 2023Rethinking and Simplifying Bootstrapped Graph Latentscited by 10

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.