mlsanityData to 7 Oct 2026

Self-supervised Graph-level Representation Learning with Local and Global Structure

arXiv 2106.04113 ↗First posted 8 Jun 2021cs.LGcited by 125

Not on the map. Why?

Abstract (arXiv)

This paper studies unsupervised/self-supervised whole-graph representation learning, which is critical in many tasks such as molecule properties prediction in drug and material discovery. Existing methods mainly focus on preserving the local similarity structure between different graph instances but fail to discover the global semantic structure of the entire data set. In this paper, we propose a unified framework called Local-instance and Global-semantic Learning (GraphLoG) for self-supervised whole-graph representation learning. Specifically, besides preserving the local similarities, GraphLoG introduces the hierarchical prototypes to capture the global semantic clusters. An efficient online expectation-maximization (EM) algorithm is further developed for learning the model. We evaluate GraphLoG by pre-training it on massive unlabeled graphs followed by fine-tuning on downstream tasks. Extensive experiments on both chemical and biological benchmark data sets demonstrate the effectiveness of the proposed approach.

Key facts. Cited by 125 papers in the map (data through 7 Oct 2026). Builds on 33 of its 67 references that are in the map. First posted on arXiv 8 Jun 2021.

Builds on

33 of 67 references.

References · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2017Graph Attention Networkscited by 8,854
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2017Inductive Representation Learning on Large Graphscited by 6,284
  8. 2018How Powerful are Graph Neural Networks?cited by 4,355
  9. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  10. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  11. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  12. 2016Variational Graph Auto-Encoderscited by 1,373
21 more
  1. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  2. 2018Deep Graph Infomaxcited by 1,101
  3. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  4. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 786
  5. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 763
  6. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
  7. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
  8. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
  9. 2015LINE: Large-scale Information Network Embeddingcited by 582
  10. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 461
  11. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 454
  12. 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
  13. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 302
  14. 2016Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clusteringcited by 237
  15. 2017graph2vec: Learning Distributed Representations of Graphscited by 228
  16. 2016Joint Unsupervised Learning of Deep Representations and Image Clusterscited by 205
  17. 2020When Does Self-Supervision Help Graph Convolutional Networks?cited by 121
  18. 2020A Graph to Graphs Framework for Retrosynthesis Predictioncited by 85
  19. 2017Learning Graph Representations with Embedding Propagationcited by 38
  20. 2018Pre-training Graph Neural Networks with Kernelscited by 8
  21. 2016AptRank: An Adaptive PageRank Model for Protein Function Prediction on Bi-relational Graphscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 32; 2023: 34; 2024: 32; 2025: 10; 2026: 7.

125 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  2. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
  3. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  4. 2022Protein Representation Learning by Geometric Structure Pretrainingcited by 161
  5. 20213D Infomax improves GNNs for Molecular Property Predictioncited by 146
  6. 2021From Canonical Correlation Analysis to Self-supervised Graph Neural Networkscited by 138
  7. 2022Universal Prompt Tuning for Graph Neural Networkscited by 100
  8. 2022Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learningcited by 86
  9. 2021GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophilycited by 84
  10. 2025SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptationcited by 61
  11. 2022Let Invariant Rationale Discovery Inspire Graph Contrastive Learningcited by 57
  12. 2023Rethinking Tokenizer and Decoder in Masked Graph Modeling for Moleculescited by 49
38 more
  1. 2022HCSC: Hierarchical Contrastive Selective Codingcited by 43
  2. 2023Generalized Graph Prompt: Toward a Unification of Pre-Training and Downstream Tasks on Graphscited by 39
  3. 2021Prototypical Graph Contrastive Learningcited by 37
  4. 2021Chemical-Reaction-Aware Molecule Representation Learningcited by 36
  5. 2024On the Scalability of GNNs for Molecular Graphscited by 36
  6. 2022Multi-task Self-supervised Graph Neural Networks Enable Stronger Task Generalizationcited by 35
  7. 2023SGL-PT: A Strong Graph Learner with Graph Prompt Tuningcited by 35
  8. 2022A Systematic Survey of Chemical Pre-trained Modelscited by 34
  9. 2021A Self-supervised Mixed-curvature Graph Neural Networkcited by 32
  10. 2024Graph Transformers: A Surveycited by 32
  11. 2022KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Predictioncited by 31
  12. 2023Graph Positional and Structural Encodercited by 31
  13. 2021Augmentations in Graph Contrastive Learning: Current Methodological Flaws & Towards Better Practicescited by 24
  14. 2022Transductive Linear Probing: A Novel Framework for Few-Shot Node Classificationcited by 24
  15. 2022A Survey of Pretraining on Graphs: Taxonomy, Methods, and Applicationscited by 23
  16. 2023SeeGera: Self-supervised Semi-implicit Graph Variational Auto-encoders with Maskingcited by 23
  17. 2022Graph Self-supervised Learning with Accurate Discrepancy Learningcited by 21
  18. 2022GLCC: A General Framework for Graph-Level Clusteringcited by 21
  19. 2022Eliciting Structural and Semantic Global Knowledge in Unsupervised Graph Contrastive Learningcited by 20
  20. 2023Fragment-based Pretraining and Finetuning on Molecular Graphscited by 20
  21. 2022Fine-Tuning Graph Neural Networks via Graph Topology induced Optimal Transportcited by 19
  22. 2023Data-Centric Learning from Unlabeled Graphs with Diffusion Modelcited by 19
  23. 2023Advancing Volumetric Medical Image Segmentation via Global-Local Masked Autoencodercited by 19
  24. 2023Motif-aware Attribute Masking for Molecular Graph Pre-trainingcited by 16
  25. 2023Pre-Training Protein Encoder via Siamese Sequence-Structure Diffusion Trajectory Predictioncited by 13
  26. 2023Search to Fine-tune Pre-trained Graph Neural Networks for Graph-level Taskscited by 13
  27. 2024TopoGCL: Topological Graph Contrastive Learningcited by 13
  28. 2023Activity Cliff Prediction: Dataset and Benchmarkcited by 11
  29. 2024Pre-training Graph Neural Networks on Molecules by Using Subgraph-Conditioned Graph Information Bottleneckcited by 11
  30. 2025ProtoMol: Enhancing Molecular Property Prediction via Prototype-Guided Multimodal Learningcited by 11
  31. 2022Decoupled Self-supervised Learning for Non-Homophilous Graphscited by 10
  32. 2024A Graph is Worth K Words: Euclideanizing Graph using Pure Transformercited by 10
  33. 2022Motif-based Graph Representation Learning with Application to Chemical Moleculescited by 9
  34. 2022Hyperbolic Hierarchical Contrastive Hashingcited by 9
  35. 2023Affinity Uncertainty-based Hard Negative Mining in Graph Contrastive Learningcited by 9
  36. 2024Subgraph-level Universal Prompt Tuningcited by 9
  37. 2024A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspectivecited by 9
  38. 2022Generative Subgraph Contrast for Self-Supervised Graph Representation Learningcited by 8

50 most cited of 125 All 125 (JSON)