mlsanityData to 7 Oct 2026

Evaluating Self-Supervised Learning for Molecular Graph Embeddings

arXiv 2206.08005 ↗First posted 16 Jun 2022cs.LG q-bio.QMcited by 22

Not on the map. Why?

Abstract (arXiv)

Graph Self-Supervised Learning (GSSL) provides a robust pathway for acquiring embeddings without expert labelling, a capability that carries profound implications for molecular graphs due to the staggering number of potential molecules and the high cost of obtaining labels. However, GSSL methods are designed not for optimisation within a specific domain but rather for transferability across a variety of downstream tasks. This broad applicability complicates their evaluation. Addressing this challenge, we present "Molecular Graph Representation Evaluation" (MOLGRAPHEVAL), generating detailed profiles of molecular graph embeddings with interpretable and diversified attributes. MOLGRAPHEVAL offers a suite of probing tasks grouped into three categories: (i) generic graph, (ii) molecular substructure, and (iii) embedding space properties. By leveraging MOLGRAPHEVAL to benchmark existing GSSL methods against both current downstream datasets and our suite of tasks, we uncover significant inconsistencies between inferences drawn solely from existing datasets and those derived from more nuanced probing. These findings suggest that current evaluation methodologies fail to capture the entirety of the landscape.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 53 of its 128 references that are in the map. First posted on arXiv 16 Jun 2022.

Builds on

53 of 128 references.

References · 53

  1. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2017Inductive Representation Learning on Large Graphscited by 6,284
  4. 2018How Powerful are Graph Neural Networks?cited by 4,355
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  6. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  7. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  8. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  9. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  10. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  11. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  12. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
38 more
  1. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  2. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
  3. 2020Self-supervised Learning: Generative or Contrastivecited by 693
  4. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
  5. 2020Principal Neighbourhood Aggregation for Graph Netscited by 477
  6. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
  7. 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
  8. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 382
  9. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
  10. 2019Analyzing Learned Molecular Representations for Property Predictioncited by 373
  11. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  12. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 302
  13. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  14. 2021Graph Self-Supervised Learning: A Surveycited by 289
  15. 2021Graph Contrastive Learning Automatedcited by 287
  16. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  17. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 240
  18. 2020GEOM: Energy-annotated molecular conformations for property prediction and molecular generationcited by 238
  19. 2020Can Graph Neural Networks Count Substructures?cited by 235
  20. 2019Evaluating Protein Transfer Learning with TAPEcited by 229
  21. 2018Characterizing and Avoiding Negative Transfercited by 226
  22. 2021ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Predictioncited by 204
  23. 2020Unsupervised Point Cloud Pre-Training via Occlusion Completioncited by 186
  24. 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
  25. 2022Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pocketscited by 182
  26. 2020ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computingcited by 162
  27. 2021On Feature Decorrelation in Self-Supervised Learningcited by 160
  28. 2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
  29. 20213D Infomax improves GNNs for Molecular Property Predictioncited by 146
  30. 2022EquiBind: Geometric Deep Learning for Drug Binding Structure Predictioncited by 143
  31. 2021Self-supervised Learning on Graphs: Contrastive, Generative,or Predictivecited by 136
  32. 2021An Empirical Study of Graph Contrastive Learningcited by 113
  33. 2018N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Moleculescited by 93
  34. 2020ATOM3D: Tasks On Molecules in Three Dimensionscited by 73
  35. 2021Decoupling the Role of Data, Attention, and Losses in Multimodal Transformerscited by 63
  36. 2021Graph Robustness Benchmark: Benchmarking the Adversarial Robustness of Graph Machine Learningcited by 44
  37. 2021Prototypical Graph Contrastive Learningcited by 37
  38. 2022MGAE: Masked Autoencoders for Self-Supervised Learning on Graphscited by 27

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 7; 2024: 4; 2025: 5; 2026: 4.

22 papers.

Sort:

  1. 2023Challenges and Applications of Large Language Modelscited by 251
  2. 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 165
  3. 2023Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Languagecited by 44
  4. 2023A Group Symmetric Stochastic Differential Equation Model for Molecule Multi-modal Pretrainingcited by 32
  5. 2023Graph Positional and Structural Encodercited by 31
  6. 2023Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materialscited by 22
  7. 2024Progress and Opportunities of Foundation Models in Bioinformaticscited by 13
  8. 2023Probing Graph Representationscited by 10
  9. 2024Learning Molecular Representation in a Cellcited by 8
  10. 2023Improving Self-supervised Molecular Representation Learning using Persistent Homologycited by 6
  11. 2024Efficient Fine-Tuning of Single-Cell Foundation Models Enables Zero-Shot Molecular Perturbation Predictioncited by 3
  12. 2022Flaky Performances when Pretraining on Relational Databasescited by 2
10 more
  1. 2025Supervised Pretraining for Material Property Predictioncited by 1
  2. 2025Self-Supervised Learning on Molecular Graphs: A Systematic Investigation of Masking Designcited by 1
  3. 2024Range-aware Positional Encoding via High-order Pretraining: Theory and Practicecited by 0
  4. 2025Learning the Neighborhood: Contrast-Free Multimodal Self-Supervised Molecular Graph Pretrainingcited by 0
  5. 2025Optimizing Classification of Infrequent Labels by Reducing Variability in Label Distributioncited by 0
  6. 2025Hierarchical Structure-Property Alignment for Data-Efficient Molecular Generation and Editingcited by 0
  7. 2026Chem-GMNet: A Sphere-Native Geometric Transformer for Molecular Property Predictioncited by 0
  8. 2026Periodic Topological Deep Learning for Polymer Design and Discoverycited by 0
  9. 2026Communicability-Inspired Positional Encoding (CIPE)cited by 0
  10. 2026Towards Multi-Label Graph Foundation Models: from Single-Vector Representation Learning to Multi-Semantic Basis Learningcited by 0