mlsanityData to 7 Oct 2026

What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoders

arXiv 2205.10053 ↗First posted 20 May 2022cs.LG cs.AIcited by 82

Not on the map. Why?

Abstract (arXiv)

The last years have witnessed the emergence of a promising self-supervised learning strategy, referred to as masked autoencoding. However, there is a lack of theoretical understanding of how masking matters on graph autoencoders (GAEs). In this work, we present masked graph autoencoder (MaskGAE), a self-supervised learning framework for graph-structured data. Different from standard GAEs, MaskGAE adopts masked graph modeling (MGM) as a principled pretext task - masking a portion of edges and attempting to reconstruct the missing part with partially visible, unmasked graph structure. To understand whether MGM can help GAEs learn better representations, we provide both theoretical and empirical evidence to comprehensively justify the benefits of this pretext task. Theoretically, we establish close connections between GAEs and contrastive learning, showing that MGM significantly improves the self-supervised learning scheme of GAEs. Empirically, we conduct extensive experiments on a variety of graph benchmarks, demonstrating the superiority of MaskGAE over several state-of-the-arts on both link prediction and node classification tasks.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). Builds on 38 of its 54 references that are in the map. First posted on arXiv 20 May 2022.

Builds on

38 of 54 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2017Graph Attention Networkscited by 8,854
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  7. 2017Inductive Representation Learning on Large Graphscited by 6,284
  8. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  9. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  10. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  11. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  12. 2016Variational Graph Auto-Encoderscited by 1,373
26 more
  1. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  2. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  3. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  4. 2018Deep Graph Infomaxcited by 1,101
  5. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  6. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
  7. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  8. 2018MINE: Mutual Information Neural Estimationcited by 680
  9. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  10. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
  11. 2019On Variational Bounds of Mutual Informationcited by 550
  12. 2020Deep Graph Contrastive Representation Learningcited by 535
  13. 2020Graph Contrastive Learning with Adaptive Augmentationcited by 519
  14. 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 299
  15. 2021Graph Contrastive Learning Automatedcited by 287
  16. 2020Graph Representation Learning via Graphical Mutual Information Maximizationcited by 217
  17. 2020Generalization and Representational Limits of Graph Neural Networkscited by 217
  18. 2020Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learningcited by 148
  19. 2021From Canonical Correlation Analysis to Self-supervised Graph Neural Networkscited by 138
  20. 2021Self-supervised Learning on Graphs: Contrastive, Generative,or Predictivecited by 136
  21. 2021Augmentation-Free Self-Supervised Learning on Graphscited by 94
  22. 2018Adversarially Regularized Graph Autoencoder for Graph Embeddingcited by 84
  23. 2021A Review on Graph Neural Network Methods in Financial Applicationscited by 77
  24. 2021Understanding Structural Vulnerability in Graph Convolutional Networkscited by 37
  25. 2022MGAE: Masked Autoencoders for Self-Supervised Learning on Graphscited by 27
  26. 2020Neural Methods for Point-wise Dependency Estimationcited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 28; 2025: 27; 2026: 17.

82 papers.

Sort:

By area:

  1. 2023Graph Prompt Learning: A Comprehensive Survey and Beyondcited by 61
  2. 2025SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptationcited by 61
  3. 2023Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Modelscited by 37
  4. 2025GraphPFN: A Prior-Data Fitted Graph Foundation Modelcited by 20
  5. 2024UniCorn: A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learningcited by 18
  6. 2024Revisiting Modularity Maximization for Graph Clustering: A Contrastive Learning Perspectivecited by 18
  7. 2024State Space Models on Temporal Graphs: A First-Principles Studycited by 16
  8. 2024HypeBoy: Generative Self-Supervised Representation Learning on Hypergraphscited by 15
  9. 2024UniGLM: Training One Unified Language Model for Text-Attributed Graph Embeddingcited by 14
  10. 2024A Survey on Learning from Graphs with Heterophily: Recent Advances and Future Directionscited by 13
  11. 2024Beyond Redundancy: Information-aware Unsupervised Multiplex Graph Structure Learningcited by 12
  12. 2024Masked Graph Autoencoder with Non-discrete Bandwidthscited by 11
38 more
  1. 2024Noise-Resilient Unsupervised Graph Representation Learning via Multi-Hop Feature Quality Estimationcited by 11
  2. 2025Generative Graph Pattern Machinecited by 11
  3. 2023RARE: Robust Masked Graph Autoencodercited by 10
  4. 2024A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspectivecited by 9
  5. 2024One Fits All: Learning Fair Graph Neural Networks for Various Sensitive Attributescited by 9
  6. 2025Circuit Representation Learning with Masked Gate Modeling and Verilog-AIG Alignmentcited by 8
  7. 2024Where to Mask: Structure-Guided Masking for Graph Masked Autoencoderscited by 7
  8. 2023A Graph is Worth 1-bit Spikes: When Graph Contrastive Learning Meets Spiking Neural Networkscited by 6
  9. 2024A Comprehensive Survey on Self-Supervised Learning for Recommendationcited by 6
  10. 2024A Survey of Few-Shot Learning on Graphs: from Meta-Learning to Pre-Training and Prompt Learningcited by 5
  11. 2025Hierarchical Vector Quantized Graph Autoencoder with Annealing-Based Code Selectioncited by 5
  12. 2023Revealing the Power of Masked Autoencoders in Traffic Forecastingcited by 4
  13. 2023Hetero2Net: Heterophily-aware Representation Learning on Heterogenerous Graphscited by 4
  14. 2025CIMAGE: Exploiting the Conditional Independence in Masked Graph Auto-encoderscited by 4
  15. 2025Boosting Bot Detection via Heterophily-Aware Representation Learning and Prototype-Guided Cluster Discoverycited by 4
  16. 2025Discrepancy-Aware Graph Mask Auto-Encodercited by 4
  17. 2024MOOSS: Mask-Enhanced Temporal Contrastive Learning for Smooth State Evolution in Visual Reinforcement Learningcited by 3
  18. 2025Balancing Graph Embedding Smoothness in Self-Supervised Learning via Information-Theoretic Decompositioncited by 3
  19. 2025Aggregation Buffer: Revisiting DropEdge with a New Parameter Blockcited by 3
  20. 2025Graph Positional Autoencoders as Self-supervised Learnerscited by 3
  21. 2025Graph Prompting for Graph Learning Models: Recent Advances and Future Directionscited by 3
  22. 2023A Generative Self-Supervised Framework using Functional Connectivity in fMRI Datacited by 2
  23. 2024Exploring Task Unification in Graph Representation Learning via Generative Approachcited by 2
  24. 2024textttSynC: Synergistic Boosting of Structure and Representation for Deep Graph Clusteringcited by 2
  25. 2025Toward a Graph Foundation Model: Pre-Training Transformers With Random Walkscited by 2
  26. 2025GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learningcited by 2
  27. 2026LEDA: Latent Semantic Distribution Alignment for Multi-domain Graph Pre-trainingcited by 2
  28. 2026Node4All: Learning Node Representation Beyond Datasetscited by 2
  29. 2023Cross-View Graph Consistency Learning for Invariant Graph Representationscited by 1
  30. 2024Variational Graph Auto-Encoder Based Inductive Learning Method for Semi-Supervised Classificationcited by 1
  31. 2024Hi-GMAE: Hierarchical Graph Masked Autoencoderscited by 1
  32. 2024Disentangled Generative Graph Representation Learningcited by 1
  33. 2024Revisiting Graph Autoencoders as Implicit Contrastive Learnerscited by 1
  34. 2025FedRIR: Rethinking Information Representation in Federated Learningcited by 1
  35. 2025GMLM: Bridging Graph Neural Networks and Language Models for Heterophilic Node Classificationcited by 1
  36. 2025Graffe: Graph Representation Learning via Diffusion Probabilistic Modelscited by 1
  37. 2025Beyond Tokens: Enhancing RTL Quality Estimation via Structural Graph Learningcited by 1
  38. 2026Exploiting Label-Independent Regularization from Spatial Dependencies for Whole Slide Image Analysiscited by 1

50 most cited of 82 All 82 (JSON)