# What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoders

arXiv: https://arxiv.org/abs/2205.10053 · HTML: https://mlsanity.com/paper/2205.10053/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 54 references that are in the map.
- First posted on arXiv 20 May 2022.

## Builds on (most cited first, 30 of 38)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [DropEdge: Towards Deep Graph Convolutional Networks on Node Classification](https://mlsanity.com/paper/1907.10903/) (2019)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [Deep Graph Contrastive Representation Learning](https://mlsanity.com/paper/2006.04131/) (2020)
- [Graph Contrastive Learning with Adaptive Augmentation](https://mlsanity.com/paper/2010.14945/) (2020)
- [Information-theoretic analysis of generalization capability of learning algorithms](https://mlsanity.com/paper/1705.07809/) (2017)
- [Graph Contrastive Learning Automated](https://mlsanity.com/paper/2106.07594/) (2021)
- [Generalization and Representational Limits of Graph Neural Networks](https://mlsanity.com/paper/2002.06157/) (2020)
- [Graph Representation Learning via Graphical Mutual Information Maximization](https://mlsanity.com/paper/2002.01169/) (2020)
- [Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning](https://mlsanity.com/paper/2010.16103/) (2020)

## Built on by (most cited first, 30 of 82)

- [SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptation](https://mlsanity.com/paper/2502.05424/) (2025)
- [Graph Prompt Learning: A Comprehensive Survey and Beyond](https://mlsanity.com/paper/2311.16534/) (2023)
- [Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models](https://mlsanity.com/paper/2310.03965/) (2023)
- [GraphPFN: A Prior-Data Fitted Graph Foundation Model](https://mlsanity.com/paper/2509.21489/) (2025)
- [UniCorn: A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learning](https://mlsanity.com/paper/2405.10343/) (2024)
- [Revisiting Modularity Maximization for Graph Clustering: A Contrastive Learning Perspective](https://mlsanity.com/paper/2406.14288/) (2024)
- [HypeBoy: Generative Self-Supervised Representation Learning on Hypergraphs](https://mlsanity.com/paper/2404.00638/) (2024)
- [State Space Models on Temporal Graphs: A First-Principles Study](https://mlsanity.com/paper/2406.00943/) (2024)
- [UniGLM: Training One Unified Language Model for Text-Attributed Graph Embedding](https://mlsanity.com/paper/2406.12052/) (2024)
- [A Survey on Learning from Graphs with Heterophily: Recent Advances and Future Directions](https://mlsanity.com/paper/2401.09769/) (2024)
- [Beyond Redundancy: Information-aware Unsupervised Multiplex Graph Structure Learning](https://mlsanity.com/paper/2409.17386/) (2024)
- [Masked Graph Autoencoder with Non-discrete Bandwidths](https://mlsanity.com/paper/2402.03814/) (2024)
- [Noise-Resilient Unsupervised Graph Representation Learning via Multi-Hop Feature Quality Estimation](https://mlsanity.com/paper/2407.19944/) (2024)
- [Generative Graph Pattern Machine](https://mlsanity.com/paper/2505.16130/) (2025)
- [RARE: Robust Masked Graph Autoencoder](https://mlsanity.com/paper/2304.01507/) (2023)
- [A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective](https://mlsanity.com/paper/2403.16137/) (2024)
- [One Fits All: Learning Fair Graph Neural Networks for Various Sensitive Attributes](https://mlsanity.com/paper/2406.13544/) (2024)
- [Circuit Representation Learning with Masked Gate Modeling and Verilog-AIG Alignment](https://mlsanity.com/paper/2502.12732/) (2025)
- [Where to Mask: Structure-Guided Masking for Graph Masked Autoencoders](https://mlsanity.com/paper/2404.15806/) (2024)
- [A Graph is Worth 1-bit Spikes: When Graph Contrastive Learning Meets Spiking Neural Networks](https://mlsanity.com/paper/2305.19306/) (2023)
- [A Survey of Few-Shot Learning on Graphs: from Meta-Learning to Pre-Training and Prompt Learning](https://mlsanity.com/paper/2402.01440/) (2024)
- [A Comprehensive Survey on Self-Supervised Learning for Recommendation](https://mlsanity.com/paper/2404.03354/) (2024)
- [Revealing the Power of Masked Autoencoders in Traffic Forecasting](https://mlsanity.com/paper/2309.15169/) (2023)
- [Hierarchical Vector Quantized Graph Autoencoder with Annealing-Based Code Selection](https://mlsanity.com/paper/2504.12715/) (2025)
- [Hetero2Net: Heterophily-aware Representation Learning on Heterogenerous Graphs](https://mlsanity.com/paper/2310.11664/) (2023)
- [CIMAGE: Exploiting the Conditional Independence in Masked Graph Auto-encoders](https://mlsanity.com/paper/2503.07852/) (2025)
- [Boosting Bot Detection via Heterophily-Aware Representation Learning and Prototype-Guided Cluster Discovery](https://mlsanity.com/paper/2506.00989/) (2025)
- [Discrepancy-Aware Graph Mask Auto-Encoder](https://mlsanity.com/paper/2506.19343/) (2025)
- [MOOSS: Mask-Enhanced Temporal Contrastive Learning for Smooth State Evolution in Visual Reinforcement Learning](https://mlsanity.com/paper/2409.02714/) (2024)
- [Balancing Graph Embedding Smoothness in Self-Supervised Learning via Information-Theoretic Decomposition](https://mlsanity.com/paper/2504.12011/) (2025)

## Abstract (arXiv)

The last years have witnessed the emergence of a promising self-supervised learning strategy, referred to as masked autoencoding. However, there is a lack of theoretical understanding of how masking matters on graph autoencoders (GAEs). In this work, we present masked graph autoencoder (MaskGAE), a self-supervised learning framework for graph-structured data. Different from standard GAEs, MaskGAE adopts masked graph modeling (MGM) as a principled pretext task - masking a portion of edges and attempting to reconstruct the missing part with partially visible, unmasked graph structure. To understand whether MGM can help GAEs learn better representations, we provide both theoretical and empirical evidence to comprehensively justify the benefits of this pretext task. Theoretically, we establish close connections between GAEs and contrastive learning, showing that MGM significantly improves the self-supervised learning scheme of GAEs. Empirically, we conduct extensive experiments on a variety of graph benchmarks, demonstrating the superiority of MaskGAE over several state-of-the-arts on both link prediction and node classification tasks.