Self-Supervised Learning on Molecular Graphs: A Systematic Investigation of Masking Design
Not on the map. Why?
Abstract (arXiv)
Self-supervised learning (SSL) plays a central role in molecular representation learning. Yet, many recent innovations in masking-based pretraining are introduced as heuristics and lack principled evaluation, obscuring which design choices are genuinely effective. This work cast the entire pretrain-finetune workflow into a unified probabilistic framework, enabling a transparent comparison and deeper understanding of masking strategies. Building on this formalism, we conduct a controlled study of three core design dimensions: masking distribution, prediction target, and encoder architecture, under rigorously controlled settings. We further employ information-theoretic measures to assess the informativeness of pretraining signals and connect them to empirically benchmarked downstream performance. Our findings reveal a surprising insight: sophisticated masking distributions offer no consistent benefit over uniform sampling for common node-level prediction tasks. Instead, the choice of prediction target and its synergy with the encoder architecture are far more critical. Specifically, shifting to semantically richer targets yields substantial downstream improvements, particularly when paired with expressive Graph Transformer encoders. These insights offer practical guidance for developing more effective SSL methods for molecular graphs.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 26 of its 43 references that are in the map. First posted on arXiv 8 Dec 2025.
Builds on
26 of 43 references.
References · 26
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2017Neural Discrete Representation Learningcited by 4,650
- 2018How Powerful are Graph Neural Networks?cited by 4,355
- 2017Neural Message Passing for Quantum Chemistrycited by 3,460
- 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
- 2020Graph Contrastive Learning with Augmentationscited by 1,203
- 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
- 2019Strategies for Pre-training Graph Neural Networkscited by 964
- 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
- 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
- 2020Self-supervised Learning: Generative or Contrastivecited by 693
14 more
- 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 656
- 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
- 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
- 2021Motif-based Graph Self-Supervised Learning for Molecular Property Predictioncited by 159
- 2023Rethinking Tokenizer and Decoder in Masked Graph Modeling for Moleculescited by 49
- 2022Evaluating Self-Supervised Learning for Molecular Graph Embeddingscited by 22
- 2023Motif-aware Attribute Masking for Molecular Graph Pre-trainingcited by 16
- 2024An end-to-end attention-based approach for learning on graphscited by 11
- 2023Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learningcited by 9
- 2024Where to Mask: Structure-Guided Masking for Graph Masked Autoencoderscited by 7
- 2025Equivariant Masked Position Prediction for Efficient Molecular Representationcited by 4
- 2024SMI-Editor: Edit-based SMILES Language Model with Fragment-level Supervisioncited by 3
- 2024UGMAE: A Unified Framework for Graph Masked Autoencoderscited by 2
- 2024MaskMol: Knowledge-guided Molecular Image Pre-Training Framework for Activity Cliffscited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.