# SeeGera: Self-supervised Semi-implicit Graph Variational Auto-encoders with Masking

arXiv: https://arxiv.org/abs/2301.12458 · HTML: https://mlsanity.com/paper/2301.12458/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 43 references that are in the map.
- First posted on arXiv 29 Jan 2023.

## Builds on (most cited first, 30 of 35)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [Deep Graph Contrastive Representation Learning](https://mlsanity.com/paper/2006.04131/) (2020)
- [Graph Contrastive Learning with Adaptive Augmentation](https://mlsanity.com/paper/2010.14945/) (2020)
- [GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training](https://mlsanity.com/paper/2006.09963/) (2020)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [GraphMAE: Self-Supervised Masked Graph Autoencoders](https://mlsanity.com/paper/2205.10803/) (2022)
- [InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization](https://mlsanity.com/paper/1908.01000/) (2019)
- [GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models](https://mlsanity.com/paper/1802.08773/) (2018)
- [Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation](https://mlsanity.com/paper/1806.02473/) (2018)
- [Ladder Variational Autoencoders](https://mlsanity.com/paper/1602.02282/) (2016)
- [GPT-GNN: Generative Pre-Training of Graph Neural Networks](https://mlsanity.com/paper/2006.15437/) (2020)
- [Graph Self-Supervised Learning: A Survey](https://mlsanity.com/paper/2103.00111/) (2021)
- [Large-Scale Representation Learning on Graphs via Bootstrapping](https://mlsanity.com/paper/2102.06514/) (2021)
- [Graph-Bert: Only Attention is Needed for Learning Graph Representations](https://mlsanity.com/paper/2001.05140/) (2020)
- [Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning](https://mlsanity.com/paper/2103.00113/) (2021)
- [From Canonical Correlation Analysis to Self-supervised Graph Neural Networks](https://mlsanity.com/paper/2106.12484/) (2021)
- [Self-supervised Graph-level Representation Learning with Local and Global Structure](https://mlsanity.com/paper/2106.04113/) (2021)
- [When Does Self-Supervision Help Graph Convolutional Networks?](https://mlsanity.com/paper/2006.09136/) (2020)
- [An Empirical Study of Graph Contrastive Learning](https://mlsanity.com/paper/2109.01116/) (2021)
- [Symmetric Graph Convolutional Autoencoder for Unsupervised Graph Representation Learning](https://mlsanity.com/paper/1908.02441/) (2019)
- [Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning](https://mlsanity.com/paper/2009.10273/) (2020)
- [Adversarially Regularized Graph Autoencoder for Graph Embedding](https://mlsanity.com/paper/1802.04407/) (2018)

## Built on by (most cited first, 23 of 23)

- [Rethinking Graph Masked Autoencoders through Alignment and Uniformity](https://mlsanity.com/paper/2402.07225/) (2024)
- [A Survey on Learning from Graphs with Heterophily: Recent Advances and Future Directions](https://mlsanity.com/paper/2401.09769/) (2024)
- [Masked Graph Autoencoder with Non-discrete Bandwidths](https://mlsanity.com/paper/2402.03814/) (2024)
- [RARE: Robust Masked Graph Autoencoder](https://mlsanity.com/paper/2304.01507/) (2023)
- [Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning](https://mlsanity.com/paper/2310.15523/) (2023)
- [A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective](https://mlsanity.com/paper/2403.16137/) (2024)
- [PSP: Pre-Training and Structure Prompt Tuning for Graph Neural Networks](https://mlsanity.com/paper/2310.17394/) (2023)
- [Exploring Correlations of Self-Supervised Tasks for Graphs](https://mlsanity.com/paper/2405.04245/) (2024)
- [Hierarchical Vector Quantized Graph Autoencoder with Annealing-Based Code Selection](https://mlsanity.com/paper/2504.12715/) (2025)
- [CIMAGE: Exploiting the Conditional Independence in Masked Graph Auto-encoders](https://mlsanity.com/paper/2503.07852/) (2025)
- [Discrepancy-Aware Graph Mask Auto-Encoder](https://mlsanity.com/paper/2506.19343/) (2025)
- [Graph Positional Autoencoders as Self-supervised Learners](https://mlsanity.com/paper/2505.23345/) (2025)
- [Graph Ranking Contrastive Learning: A Extremely Simple yet Efficient Method](https://mlsanity.com/paper/2310.14525/) (2023)
- [Exploring Task Unification in Graph Representation Learning via Generative Approach](https://mlsanity.com/paper/2403.14340/) (2024)
- [Context-Enhanced Multi-View Trajectory Representation Learning: Bridging the Gap through Self-Supervised Models](https://mlsanity.com/paper/2410.13196/) (2024)
- [Variational Graph Autoencoder for Heterogeneous Information Networks with Missing and Inaccurate Attributes](https://mlsanity.com/paper/2311.07929/) (2023)
- [Semantic Evolvement Enhanced Graph Autoencoder for Rumor Detection](https://mlsanity.com/paper/2404.16076/) (2024)
- [Hi-GMAE: Hierarchical Graph Masked Autoencoders](https://mlsanity.com/paper/2405.10642/) (2024)
- [Disentangled Generative Graph Representation Learning](https://mlsanity.com/paper/2408.13471/) (2024)
- [Revisiting Graph Autoencoders as Implicit Contrastive Learners](https://mlsanity.com/paper/2410.10241/) (2024)
- [Preserving Node Distinctness in Graph Autoencoders via Similarity Distillation](https://mlsanity.com/paper/2406.17517/) (2024)
- [Informative Subgraphs Aware Masked Auto-Encoder in Dynamic Graphs](https://mlsanity.com/paper/2409.09262/) (2024)
- [IMPA-HGAE:Intra-Meta-Path Augmented Heterogeneous Graph Autoencoder](https://mlsanity.com/paper/2506.06809/) (2025)

## Abstract (arXiv)

Generative graph self-supervised learning (SSL) aims to learn node representations by reconstructing the input graph data. However, most existing methods focus on unsupervised learning tasks only and very few work has shown its superiority over the state-of-the-art graph contrastive learning (GCL) models, especially on the classification task. While a very recent model has been proposed to bridge the gap, its performance on unsupervised learning tasks is still unknown. In this paper, to comprehensively enhance the performance of generative graph SSL against other GCL models on both unsupervised and supervised learning tasks, we propose the SeeGera model, which is based on the family of self-supervised variational graph auto-encoder (VGAE). Specifically, SeeGera adopts the semi-implicit variational inference framework, a hierarchical variational framework, and mainly focuses on feature reconstruction and structure/feature masking. On the one hand, SeeGera co-embeds both nodes and features in the encoder and reconstructs both links and features in the decoder. Since feature embeddings contain rich semantic information on features, they can be combined with node embeddings to provide fine-grained knowledge for feature reconstruction. On the other hand, SeeGera adds an additional layer for structure/feature masking to the hierarchical variational framework, which boosts the model generalizability. We conduct extensive experiments comparing SeeGera with 9 other state-of-the-art competitors. Our results show that SeeGera can compare favorably against other state-of-the-art GCL methods in a variety of unsupervised and supervised learning tasks.