mlsanityData to 14 Oct 2026

GraphMAE: Self-Supervised Masked Graph Autoencoders

arXiv 2205.10803 ↗First posted 22 May 2022cs.LGcited by 448

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) has been extensively explored in recent years. Particularly, generative SSL has seen emerging success in natural language processing and other AI fields, such as the wide adoption of BERT and GPT. Despite this, contrastive learning-which heavily relies on structural data augmentation and complicated training strategies-has been the dominant approach in graph SSL, while the progress of generative SSL on graphs, especially graph autoencoders (GAEs), has thus far not reached the potential as promised in other fields. In this paper, we identify and examine the issues that negatively impact the development of GAEs, including their reconstruction objective, training robustness, and error metric. We present a masked graph autoencoder GraphMAE that mitigates these issues for generative self-supervised graph pretraining. Instead of reconstructing graph structures, we propose to focus on feature reconstruction with both a masking strategy and scaled cosine error that benefit the robust training of GraphMAE. We conduct extensive experiments on 21 public datasets for three different graph learning tasks. The results manifest that GraphMAE-a simple graph autoencoder with careful designs-can consistently generate outperformance over both contrastive and generative state-of-the-art baselines. This study provides an understanding of graph autoencoders and demonstrates the potential of generative self-supervised pre-training on graphs.

Key facts. Cited by 448 papers in the map (data through 14 Oct 2026). Builds on 52 of its 63 references that are in the map. First posted on arXiv 22 May 2022.

Builds on

52 of 63 references.

References · 52

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2017Graph Attention Networkscited by 8,923
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2017Inductive Representation Learning on Large Graphscited by 6,320
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  9. 2018How Powerful are Graph Neural Networks?cited by 4,393
  10. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  11. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  12. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
38 more
  1. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  2. 2016node2vec: Scalable Feature Learning for Networkscited by 1,918
  3. 2014DeepWalk: Online Learning of Social Representationscited by 1,547
  4. 2016Variational Graph Auto-Encoderscited by 1,382
  5. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  6. 2018Deep Graph Infomaxcited by 1,106
  7. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  8. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 976
  9. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  10. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 787
  11. 2017Focal Loss for Dense Object Detectioncited by 752
  12. 2020Self-supervised Learning: Generative or Contrastivecited by 694
  13. 2020Contrastive Multi-View Representation Learning on Graphscited by 652
  14. 2015LINE: Large-scale Information Network Embeddingcited by 582
  15. 2020Deep Graph Contrastive Representation Learningcited by 539
  16. 2020Graph Contrastive Learning with Adaptive Augmentationcited by 519
  17. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 463
  18. 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
  19. 2018GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Modelscited by 413
  20. 2018Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generationcited by 364
  21. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 303
  22. 2021Graph Contrastive Learning Automatedcited by 287
  23. 2017graph2vec: Learning Distributed Representations of Graphscited by 230
  24. 2021Large-Scale Representation Learning on Graphs via Bootstrappingcited by 186
  25. 2021Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillationcited by 152
  26. 2021From Canonical Correlation Analysis to Self-supervised Graph Neural Networkscited by 141
  27. 2021Self-supervised Graph-level Representation Learning with Local and Global Structurecited by 125
  28. 2020When Does Self-Supervision Help Graph Convolutional Networks?cited by 121
  29. 2020Self-supervised Learning on Graphs: Deep Insights and New Directioncited by 108
  30. 2021InfoGCL: Information-Aware Graph Contrastive Learningcited by 105
  31. 2019Symmetric Graph Convolutional Autoencoder for Unsupervised Graph Representation Learningcited by 94
  32. 2020Adaptive Graph Encoder for Attributed Graph Embeddingcited by 89
  33. 2018Adversarially Regularized Graph Autoencoder for Graph Embeddingcited by 84
  34. 2020Contrastive Self-supervised Learning for Graph Classificationcited by 68
  35. 2019Graph Attention Auto-Encoderscited by 41
  36. 2017Learning Graph Representations with Embedding Propagationcited by 38
  37. 2022Graph Auto-Encoder Via Neighborhood Wasserstein Reconstructioncited by 28
  38. 2021Self-supervised Graph Neural Networks without explicit negative samplingcited by 25

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 15; 2023: 81; 2024: 138; 2025: 100; 2026: 114.

448 papers.

Sort:

By area:

  1. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  2. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 226
  3. 2023All in One: Multi-task Prompting for Graph Neural Networkscited by 148
  4. 2023GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learnercited by 132
  5. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 103
  6. 2024Position: Graph Foundation Models are Already Herecited by 86
  7. 2024All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretrainingcited by 86
  8. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 85
  9. 2024ZeroG: Investigating Cross-dataset Zero-shot Transferability in Graphscited by 85
  10. 2024GFT: Graph Foundation Model with Transferable Tree Vocabularycited by 85
  11. 2023GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Textcited by 82
  12. 2024UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphscited by 66
38 more
  1. 2023Graph Mixture of Experts: Learning on Large-Scale Graphs with Explicit Diversity Modelingcited by 64
  2. 2023Ti-MAE: Self-Supervised Masked Time Series Autoencoderscited by 63
  3. 2025SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptationcited by 62
  4. 2023Graph Prompt Learning: A Comprehensive Survey and Beyondcited by 61
  5. 2024LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddingscited by 61
  6. 2022A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyondcited by 58
  7. 2024GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphscited by 58
  8. 2023Rethinking Tokenizer and Decoder in Masked Graph Modeling for Moleculescited by 49
  9. 2025UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphscited by 49
  10. 2022Heterogeneous Graph Masked Autoencoderscited by 47
  11. 2024Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insightscited by 44
  12. 2023SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence Pre-trainingcited by 41
  13. 2024GLBench: A Comprehensive Benchmark for Graph with Large Language Modelscited by 36
  14. 2022Multi-task Self-supervised Graph Neural Networks Enable Stronger Task Generalizationcited by 35
  15. 2023SGL-PT: A Strong Graph Learner with Graph Prompt Tuningcited by 35
  16. 2022A Systematic Survey of Chemical Pre-trained Modelscited by 34
  17. 2023Graph Positional and Structural Encodercited by 33
  18. 2023GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learningcited by 33
  19. 2024Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlookcited by 33
  20. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  21. 2023LightPath: Lightweight and Scalable Path Representation Learningcited by 32
  22. 2024Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Treescited by 32
  23. 2023Beyond Homophily: Reconstructing Structure for Graph-agnostic Clusteringcited by 29
  24. 2024ProG: A Graph Prompt Learning Benchmarkcited by 29
  25. 2022Lifelong Embedding Learning and Transfer for Growing Knowledge Graphscited by 28
  26. 2024Foundation Models for the Electric Power Gridcited by 28
  27. 2024A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoningcited by 25
  28. 2023SeeGera: Self-supervised Semi-implicit Graph Variational Auto-encoders with Maskingcited by 23
  29. 2023Uncovering Neural Scaling Laws in Molecular Representation Learningcited by 23
  30. 2023DER-GCN: Dialogue and Event Relation-Aware Graph Convolutional Neural Network for Multimodal Dialogue Emotion Recognitioncited by 23
  31. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  32. 2022Evaluating Self-Supervised Learning for Molecular Graph Embeddingscited by 22
  33. 2023Graph-level Representation Learning with Joint-Embedding Predictive Architecturescited by 22
  34. 2024MAPE-PPI: Towards Effective and Efficient Protein-Protein Interaction Prediction via Microenvironment-Aware Protein Embeddingcited by 22
  35. 2023Multimodal Molecular Pretraining via Modality Blendingcited by 21
  36. 2023GiGaMAE: Generalizable Graph Masked Autoencoder via Collaborative Latent Space Reconstructioncited by 21
  37. 2024Contrastive Graph Representation Learning with Adversarial Cross-view Reconstruction and Information Bottleneckcited by 21
  38. 2024UniGEM: A Unified Approach to Generation and Property Prediction for Moleculescited by 21

50 most cited of 448 All 448 (JSON)