Data to 7 Oct 2026

GPT-GNN: Generative Pre-Training of Graph Neural Networks

arXiv 2006.15437 ↗First posted 27 Jun 2020cs.LG cs.SI stat.MLcited by 302

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) have been demonstrated to be powerful in modeling graph-structured data. However, training GNNs usually requires abundant task-specific labeled data, which is often arduously expensive to obtain. One effective way to reduce the labeling effort is to pre-train an expressive GNN model on unlabeled data with self-supervision and then transfer the learned model to downstream tasks with only a few labels. In this paper, we present the GPT-GNN framework to initialize GNNs by generative pre-training. GPT-GNN introduces a self-supervised attributed graph generation task to pre-train a GNN so that it can capture the structural and semantic properties of the graph. We factorize the likelihood of the graph generation into two components: 1) Attribute Generation and 2) Edge Generation. By modeling both components, GPT-GNN captures the inherent dependency between node attributes and graph structure during the generative process. Comprehensive experiments on the billion-scale Open Academic Graph and Amazon recommendation data demonstrate that GPT-GNN significantly outperforms state-of-the-art GNN models without pre-training by up to 9.1% across various downstream tasks.

Key facts. Cited by 302 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 46; 2022: 52; 2023: 75; 2024: 65; 2025: 43; 2026: 21.

302 papers.

Sort:

By area:

2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
2021Graph Self-Supervised Learning: A Surveycited by 289
2021Graph Contrastive Learning Automatedcited by 287
2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 223
2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
2021Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networkscited by 176
2022SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentationcited by 171
2023GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networkscited by 166
2021Motif-based Graph Self-Supervised Learning for Molecular Property Predictioncited by 159

38 more

2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 147
2021GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graphcited by 143
2021Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Predictioncited by 137
2021Self-supervised Learning on Graphs: Contrastive, Generative,or Predictivecited by 136
2022Data Augmentation for Deep Graph Learning: A Surveycited by 132
2023Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learningcited by 132
2023GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learnercited by 131
2021Self-supervised Graph-level Representation Learning with Local and Global Structurecited by 125
2022Pre-training via Denoising for Molecular Property Predictioncited by 124
2022A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainabilitycited by 118
2021SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networkscited by 108
2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 101
2022Universal Prompt Tuning for Graph Neural Networkscited by 100
2023PRODIGY: Enabling In-context Learning Over Graphscited by 97
2022Federated Learning on Non-IID Graphs via Structural Knowledge Sharingcited by 88
2022Graph Attention Multi-Layer Perceptroncited by 82
2023Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPTcited by 78
2022One Transformer Can Understand Both 2D & 3D Molecular Datacited by 77
2023A Comprehensive Survey on Deep Graph Representation Learningcited by 72
2022Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matchingcited by 67
2022A Survey on Graph Representation Learning Methodscited by 66
2021MoCL: Data-driven Molecular Fingerprint via Knowledge-aware Contrastive Learning from Molecular Graphcited by 65
2023Graph Mixture of Experts: Learning on Large-Scale Graphs with Explicit Diversity Modelingcited by 63
2023Graph Prompt Learning: A Comprehensive Survey and Beyondcited by 61
2025SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptationcited by 61
2022Does GNN Pretraining Help Molecular Representation?cited by 59
2022Graph Pre-training for AMR Parsing and Generationcited by 55
2023Patton: Language Model Pretraining on Text-Rich Networkscited by 49
2022Non-IID Transfer Learning on Graphscited by 43
2025Multi-Domain Graph Foundation Models: Robust Knowledge Transfer via Topology Alignmentcited by 41
2023MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphscited by 40
2022Mask and Reason: Pre-Training Knowledge Graph Transformers for Complex Logical Queriescited by 39
2023Generalized Graph Prompt: Toward a Unification of Pre-Training and Downstream Tasks on Graphscited by 39
2022Spectral Augmentation for Self-Supervised Learning on Graphscited by 38
2021Prototypical Graph Contrastive Learningcited by 37
2022Energy-Motivated Equivariant Pretraining for 3D Molecular Graphscited by 37
2021When Do Contrastive Learning Signals Help Spatio-Temporal Graph Forecasting?cited by 36
2022A Systematic Survey of Chemical Pre-trained Modelscited by 34

50 most cited of 302 All 302 (JSON)