# Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning

arXiv: https://arxiv.org/abs/2105.09111 · HTML: https://mlsanity.com/paper/2105.09111/

## Key facts

- Cited by 130 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 42 references that are in the map.
- First posted on arXiv 19 May 2021.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [Graph Transformer Networks](https://mlsanity.com/paper/1911.06455/) (2019)
- [GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training](https://mlsanity.com/paper/2006.09963/) (2020)
- [Hard Negative Mixing for Contrastive Learning](https://mlsanity.com/paper/2010.01028/) (2020)
- [MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding](https://mlsanity.com/paper/2002.01680/) (2020)
- [Graph Representation Learning via Graphical Mutual Information Maximization](https://mlsanity.com/paper/2002.01169/) (2020)
- [Heterogeneous Information Network Embedding for Recommendation](https://mlsanity.com/paper/1711.10730/) (2017)
- [GraphGAN: Graph Representation Learning with Generative Adversarial Nets](https://mlsanity.com/paper/1711.08267/) (2017)
- [Unsupervised Attributed Multiplex Network Embedding](https://mlsanity.com/paper/1911.06750/) (2019)
- [Leveraging Meta-path Contexts for Classification in Heterogeneous Information Networks](https://mlsanity.com/paper/2012.10024/) (2020)

## Built on by (most cited first, 30 of 130)

- [A Comprehensive Survey on Community Detection with Deep Learning](https://mlsanity.com/paper/2105.12584/) (2021)
- [GeomGCL: Geometric Graph Contrastive Learning for Molecular Property Prediction](https://mlsanity.com/paper/2109.11730/) (2021)
- [Graph Prompt Learning: A Comprehensive Survey and Beyond](https://mlsanity.com/paper/2311.16534/) (2023)
- [Heterogeneous Graph Masked Autoencoders](https://mlsanity.com/paper/2208.09957/) (2022)
- [HGPROMPT: Bridging Homogeneous and Heterogeneous Graphs for Few-shot Prompt Learning](https://mlsanity.com/paper/2312.01878/) (2023)
- [HiGPT: Heterogeneous Graph Language Model](https://mlsanity.com/paper/2402.16024/) (2024)
- [Contrastive Meta Learning with Behavior Multiplicity for Recommendation](https://mlsanity.com/paper/2202.08523/) (2022)
- [HINormer: Representation Learning On Heterogeneous Information Networks with Graph Transformer](https://mlsanity.com/paper/2302.11329/) (2023)
- [Graph Communal Contrastive Learning](https://mlsanity.com/paper/2110.14863/) (2021)
- [A Self-supervised Mixed-curvature Graph Neural Network](https://mlsanity.com/paper/2112.05393/) (2021)
- [HetGPT: Harnessing the Power of Prompt Tuning in Pre-Trained Heterogeneous Graph Neural Networks](https://mlsanity.com/paper/2310.15318/) (2023)
- [Self-supervised Heterogeneous Graph Pre-training Based on Structural Clustering](https://mlsanity.com/paper/2210.10462/) (2022)
- [A Survey of Deep Graph Clustering: Taxonomy, Challenge, Application, and Open Resource](https://mlsanity.com/paper/2211.12875/) (2022)
- [Towards Graph Contrastive Learning: A Survey and Beyond](https://mlsanity.com/paper/2405.11868/) (2024)
- [A Survey of Pretraining on Graphs: Taxonomy, Methods, and Applications](https://mlsanity.com/paper/2202.07893/) (2022)
- [DER-GCN: Dialogue and Event Relation-Aware Graph Convolutional Neural Network for Multimodal Dialogue Emotion Recognition](https://mlsanity.com/paper/2312.10579/) (2023)
- [CGC: Contrastive Graph Clustering for Community Detection and Tracking](https://mlsanity.com/paper/2204.08504/) (2022)
- [MAGIC: Detecting Advanced Persistent Threats via Masked Graph Representation Learning](https://mlsanity.com/paper/2310.09831/) (2023)
- [Embedding in Recommender Systems: A Survey](https://mlsanity.com/paper/2310.18608/) (2023)
- [X-GOAL: Multiplex Heterogeneous Graph Prototypical Contrastive Learning](https://mlsanity.com/paper/2109.03560/) (2021)
- [Federated Heterogeneous Graph Neural Network for Privacy-preserving Recommendation](https://mlsanity.com/paper/2310.11730/) (2023)
- [A Survey of Data-Efficient Graph Learning](https://mlsanity.com/paper/2402.00447/) (2024)
- [Inductive Cognitive Diagnosis for Fast Student Learning in Web-Based Online Intelligent Education Systems](https://mlsanity.com/paper/2404.11290/) (2024)
- [Heterogeneous Graph Contrastive Multi-view Learning](https://mlsanity.com/paper/2210.00248/) (2022)
- [Diet-ODIN: A Novel Framework for Opioid Misuse Detection with Interpretable Dietary Patterns](https://mlsanity.com/paper/2403.08820/) (2024)
- [SMiLE: Schema-augmented Multi-level Contrastive Learning for Knowledge Graph Link Prediction](https://mlsanity.com/paper/2210.04870/) (2022)
- [STERLING: Synergistic Representation Learning on Bipartite Graphs](https://mlsanity.com/paper/2302.05428/) (2023)
- [Zero-shot Item-based Recommendation via Multi-task Product Knowledge Graph Pre-Training](https://mlsanity.com/paper/2305.07633/) (2023)
- [Upper Bounding Barlow Twins: A Novel Filter for Multi-Relational Clustering](https://mlsanity.com/paper/2312.14066/) (2023)
- [When Heterophily Meets Heterogeneous Graphs: Latent Graphs Guided Unsupervised Representation Learning](https://mlsanity.com/paper/2409.00687/) (2024)

## Abstract (arXiv)

Heterogeneous graph neural networks (HGNNs) as an emerging technique have shown superior capacity of dealing with heterogeneous information network (HIN). However, most HGNNs follow a semi-supervised learning manner, which notably limits their wide use in reality since labels are usually scarce in real applications. Recently, contrastive learning, a self-supervised method, becomes one of the most exciting learning paradigms and shows great potential when there are no labels. In this paper, we study the problem of self-supervised HGNNs and propose a novel co-contrastive learning mechanism for HGNNs, named HeCo. Different from traditional contrastive learning which only focuses on contrasting positive and negative samples, HeCo employs cross-viewcontrastive mechanism. Specifically, two views of a HIN (network schema and meta-path views) are proposed to learn node embeddings, so as to capture both of local and high-order structures simultaneously. Then the cross-view contrastive learning, as well as a view mask mechanism, is proposed, which is able to extract the positive and negative embeddings from two views. This enables the two views to collaboratively supervise each other and finally learn high-level node embeddings. Moreover, two extensions of HeCo are designed to generate harder negative samples with high quality, which further boosts the performance of HeCo. Extensive experiments conducted on a variety of real-world networks show the superior performance of the proposed methods over the state-of-the-arts.