# GiGaMAE: Generalizable Graph Masked Autoencoder via Collaborative Latent Space Reconstruction

arXiv: https://arxiv.org/abs/2308.09663 · HTML: https://mlsanity.com/paper/2308.09663/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 65 references that are in the map.
- First posted on arXiv 18 Aug 2023.

## Builds on (most cited first, 30 of 44)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Revisiting Semi-Supervised Learning with Graph Embeddings](https://mlsanity.com/paper/1603.08861/) (2016)
- [Contrastive Representation Distillation](https://mlsanity.com/paper/1910.10699/) (2019)
- [Graph Neural Networks for Social Recommendation](https://mlsanity.com/paper/1902.07243/) (2019)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [Masked Feature Prediction for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2112.09133/) (2021)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [Deep Graph Contrastive Representation Learning](https://mlsanity.com/paper/2006.04131/) (2020)
- [Graph Contrastive Learning with Adaptive Augmentation](https://mlsanity.com/paper/2010.14945/) (2020)
- [GraphMAE: Self-Supervised Masked Graph Autoencoders](https://mlsanity.com/paper/2205.10803/) (2022)
- [On Mutual Information Maximization for Representation Learning](https://mlsanity.com/paper/1907.13625/) (2019)
- [GPT-GNN: Generative Pre-Training of Graph Neural Networks](https://mlsanity.com/paper/2006.15437/) (2020)
- [Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks](https://mlsanity.com/paper/2007.02901/) (2020)
- [Large-Scale Representation Learning on Graphs via Bootstrapping](https://mlsanity.com/paper/2102.06514/) (2021)

## Built on by (most cited first, 21 of 21)

- [IDEA: A Flexible Framework of Certified Unlearning for Graph Neural Networks](https://mlsanity.com/paper/2407.19398/) (2024)
- [GAugLLM: Improving Graph Contrastive Learning for Text-Attributed Graphs with Large Language Models](https://mlsanity.com/paper/2406.11945/) (2024)
- [UniGLM: Training One Unified Language Model for Text-Attributed Graph Embedding](https://mlsanity.com/paper/2406.12052/) (2024)
- [Enhancing Cognition and Explainability of Multimodal Foundation Models with Self-Synthesized Data](https://mlsanity.com/paper/2502.14044/) (2025)
- [Where to Mask: Structure-Guided Masking for Graph Masked Autoencoders](https://mlsanity.com/paper/2404.15806/) (2024)
- [GraphFM: A Comprehensive Benchmark for Graph Foundation Model](https://mlsanity.com/paper/2406.08310/) (2024)
- [CIMAGE: Exploiting the Conditional Independence in Masked Graph Auto-encoders](https://mlsanity.com/paper/2503.07852/) (2025)
- [Balancing Graph Embedding Smoothness in Self-Supervised Learning via Information-Theoretic Decomposition](https://mlsanity.com/paper/2504.12011/) (2025)
- [Graph Positional Autoencoders as Self-supervised Learners](https://mlsanity.com/paper/2505.23345/) (2025)
- [Marginal Nodes Matter: Towards Structure Fairness in Graphs](https://mlsanity.com/paper/2310.14527/) (2023)
- [UGMAE: A Unified Framework for Graph Masked Autoencoders](https://mlsanity.com/paper/2402.08023/) (2024)
- [Hi-GMAE: Hierarchical Graph Masked Autoencoders](https://mlsanity.com/paper/2405.10642/) (2024)
- [Disentangled Generative Graph Representation Learning](https://mlsanity.com/paper/2408.13471/) (2024)
- [Revisiting Graph Autoencoders as Implicit Contrastive Learners](https://mlsanity.com/paper/2410.10241/) (2024)
- [Learning Time-Varying Graph Signals via Koopman](https://mlsanity.com/paper/2511.06493/) (2025)
- [Masked AutoEncoder for Graph Clustering without Pre-defined Cluster Number k](https://mlsanity.com/paper/2401.04741/) (2024)
- [Deep Generative Models for Subgraph Prediction](https://mlsanity.com/paper/2408.04053/) (2024)
- [Informative Subgraphs Aware Masked Auto-Encoder in Dynamic Graphs](https://mlsanity.com/paper/2409.09262/) (2024)
- [Generative and Contrastive Graph Representation Learning](https://mlsanity.com/paper/2505.11776/) (2025)
- [GraphSculptor: Sculpting Pre-training Coreset for Graph Self-supervised Learning](https://mlsanity.com/paper/2605.01310/) (2026)
- [Unsupervised Graph Representation Learning with Complementary View Alignment](https://mlsanity.com/paper/2607.24338/) (2026)

## Abstract (arXiv)

Self-supervised learning with masked autoencoders has recently gained popularity for its ability to produce effective image or textual representations, which can be applied to various downstream tasks without retraining. However, we observe that the current masked autoencoder models lack good generalization ability on graph data. To tackle this issue, we propose a novel graph masked autoencoder framework called GiGaMAE. Different from existing masked autoencoders that learn node presentations by explicitly reconstructing the original graph components (e.g., features or edges), in this paper, we propose to collaboratively reconstruct informative and integrated latent embeddings. By considering embeddings encompassing graph topology and attribute information as reconstruction targets, our model could capture more generalized and comprehensive knowledge. Furthermore, we introduce a mutual information based reconstruction loss that enables the effective reconstruction of multiple targets. This learning objective allows us to differentiate between the exclusive knowledge learned from a single target and common knowledge shared by multiple targets. We evaluate our method on three downstream tasks with seven datasets as benchmarks. Extensive experiments demonstrate the superiority of GiGaMAE against state-of-the-art baselines. We hope our results will shed light on the design of foundation models on graph-structured data. Our code is available at: https://github.com/sycny/GiGaMAE.