mlsanityData to 7 Oct 2026

CPDG: A Contrastive Pre-Training Method for Dynamic Graph Neural Networks

arXiv 2307.02813 ↗First posted 6 Jul 2023cs.LG cs.SIcited by 6

Not on the map. Why?

Abstract (arXiv)

Dynamic graph data mining has gained popularity in recent years due to the rich information contained in dynamic graphs and their widespread use in the real world. Despite the advances in dynamic graph neural networks (DGNNs), the rich information and diverse downstream tasks have posed significant difficulties for the practical application of DGNNs in industrial scenarios. To this end, in this paper, we propose to address them by pre-training and present the Contrastive Pre-Training Method for Dynamic Graph Neural Networks (CPDG). CPDG tackles the challenges of pre-training for DGNNs, including generalization capability and long-short term modeling capability, through a flexible structural-temporal subgraph sampler along with structural-temporal contrastive pre-training schemes. Extensive experiments conducted on both large-scale research and industrial dynamic graph datasets show that CPDG outperforms existing methods in dynamic graph pre-training for various downstream tasks under three transfer settings.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 32 of its 60 references that are in the map. First posted on arXiv 6 Jul 2023.

Builds on

32 of 60 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2018How Powerful are Graph Neural Networks?cited by 4,355
  6. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  7. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  8. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  9. 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,134
  10. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  11. 2018Deep Graph Infomaxcited by 1,101
  12. 2021Transformers in Vision: A Surveycited by 1,009
20 more
  1. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  2. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 786
  3. 2017Deep Interest Network for Click-Through Rate Predictioncited by 541
  4. 2020Temporal Graph Networks for Deep Learning on Dynamic Graphscited by 521
  5. 2020Graph Neural Networks in Recommender Systems: A Surveycited by 493
  6. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 461
  7. 2020Inductive Representation Learning on Temporal Graphscited by 454
  8. 2019Predicting Dynamic Embedding Trajectory in Temporal Interaction Networkscited by 443
  9. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
  10. 2018Deep Learning on Graphs: A Surveycited by 393
  11. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 302
  12. 2021Inductive Representation Learning in Temporal Networks via Causal Anonymous Walkscited by 188
  13. 2023GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networkscited by 166
  14. 2022ROLAND: Graph Learning Framework for Dynamic Graphscited by 131
  15. 2020Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A surveycited by 123
  16. 2023Graph Neural Networks for temporal graphs: State of the art, open challenges, and opportunitiescited by 60
  17. 2022A Self-supervised Riemannian GNN with Time Varying Curvature for Temporal Graph Learningcited by 24
  18. 2019Deep learning in bioinformatics: introduction, application, and perspective in big data eracited by 12
  19. 2021Pre-Training on Dynamic Graph Neural Networkscited by 11
  20. 2023Graph Based Long-Term And Short-Term Interest Model for Click-Through Rate Predictioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 0; 2026: 1.

6 papers.

  1. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 118
  2. 2024A survey of dynamic graph neural networkscited by 21
  3. 2024Node-Time Conditional Prompt Learning In Dynamic Graphscited by 11
  4. 2024A Survey of Few-Shot Learning on Graphs: from Meta-Learning to Pre-Training and Prompt Learningcited by 5
  5. 2023CDR: Conservative Doubly Robust Learning for Debiased Recommendationcited by 2
  6. 2026Decoupled and Divergence-Conditioned Prompt for Multi-domain Dynamic Graph Foundation Modelscited by 0