# Attention-wise masked graph contrastive learning for predicting molecular property

arXiv: https://arxiv.org/abs/2206.08262 · HTML: https://mlsanity.com/paper/2206.08262/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 45 references that are in the map.
- First posted on arXiv 2 May 2022.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Motif-based Graph Self-Supervised Learning for Molecular Property Prediction](https://mlsanity.com/paper/2110.00987/) (2021)
- [N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Molecules](https://mlsanity.com/paper/1806.09206/) (2018)
- [Low Data Drug Discovery with One-shot Learning](https://mlsanity.com/paper/1611.03199/) (2016)
- [ASGN: An Active Semi-supervised Graph Neural Network for Molecular Property Prediction](https://mlsanity.com/paper/2007.03196/) (2020)
- [Molecular Contrastive Learning with Chemical Element Knowledge Graph](https://mlsanity.com/paper/2112.00544/) (2021)
- [Coarse-Graining Auto-Encoders for Molecular Dynamics](https://mlsanity.com/paper/1812.02706/) (2018)
- [Orbital Graph Convolutional Neural Network for Material Property Prediction](https://mlsanity.com/paper/2008.06415/) (2020)
- [Multi-View Graph Neural Networks for Molecular Property Prediction](https://mlsanity.com/paper/2005.13607/) (2020)

## Built on by (most cited first, 10 of 10)

- [Molecular Joint Representation Learning via Multi-modal Information](https://mlsanity.com/paper/2211.14042/) (2022)
- [Impact of Domain Knowledge and Multi-Modality on Intelligent Molecular Property Prediction: A Systematic Survey](https://mlsanity.com/paper/2402.07249/) (2024)
- [Survey on Recent Progress of AI for Chemistry: Methods, Applications, and Opportunities](https://mlsanity.com/paper/2502.17456/) (2025)
- [Advancements in Molecular Property Prediction: A Survey of Single and Multimodal Approaches](https://mlsanity.com/paper/2408.09461/) (2024)
- [Cross-domain feature disentanglement for interpretable modeling of tumor microenvironment impact on drug response](https://mlsanity.com/paper/2311.09264/) (2023)
- [Graph Multi-Similarity Learning for Molecular Property Prediction](https://mlsanity.com/paper/2401.17615/) (2024)
- [Interpretable Modeling of Single-cell perturbation Responses to Novel Drugs Using Cycle Consistence Learning](https://mlsanity.com/paper/2311.10315/) (2023)
- [MolPLA: A Molecular Pretraining Framework for Learning Cores, R-Groups and their Linker Joints](https://mlsanity.com/paper/2401.16771/) (2024)
- [Contrastive Dual-Interaction Graph Neural Network for Molecular Property Prediction](https://mlsanity.com/paper/2405.02628/) (2024)
- [ChemHyperMag: Physics-informed magnetic hypergraph learning improves molecular ADMET prediction](https://mlsanity.com/paper/2607.18332/) (2026)

## Abstract (arXiv)

Accurate and efficient prediction of the molecular properties of drugs is one of the fundamental problems in drug research and development. Recent advancements in representation learning have been shown to greatly improve the performance of molecular property prediction. However, due to limited labeled data, supervised learning-based molecular representation algorithms can only search limited chemical space, which results in poor generalizability. In this work, we proposed a self-supervised representation learning framework for large-scale unlabeled molecules. We developed a novel molecular graph augmentation strategy, referred to as attention-wise graph mask, to generate challenging positive sample for contrastive learning. We adopted the graph attention network (GAT) as the molecular graph encoder, and leveraged the learned attention scores as masking guidance to generate molecular augmentation graphs. By minimization of the contrastive loss between original graph and masked graph, our model can capture important molecular structure and higher-order semantic information. Extensive experiments showed that our attention-wise graph mask contrastive learning exhibit state-of-the-art performance in a couple of downstream molecular property prediction tasks.