# Molecular Contrastive Learning with Chemical Element Knowledge Graph

arXiv: https://arxiv.org/abs/2112.00544 · HTML: https://mlsanity.com/paper/2112.00544/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 49 references that are in the map.
- First posted on arXiv 1 Dec 2021.

## Builds on (most cited first, 30 of 31)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization](https://mlsanity.com/paper/1908.01000/) (2019)
- [Analyzing Learned Molecular Representations for Property Prediction](https://mlsanity.com/paper/1904.01561/) (2019)
- [Graph Contrastive Learning Automated](https://mlsanity.com/paper/2106.07594/) (2021)
- [Molecular Graph Convolutions: Moving Beyond Fingerprints](https://mlsanity.com/paper/1603.00856/) (2016)
- [Order Matters: Sequence to sequence for sets](https://mlsanity.com/paper/1511.06391/) (2015)
- [Pre-training of Graph Augmented Transformers for Medication Recommendation](https://mlsanity.com/paper/1906.00346/) (2019)
- [Multi-Stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labels](https://mlsanity.com/paper/1902.11038/) (2019)
- [Graph-based, Self-Supervised Program Repair from Diagnostic Feedback](https://mlsanity.com/paper/2005.10636/) (2020)
- [N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Molecules](https://mlsanity.com/paper/1806.09206/) (2018)
- [Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network](https://mlsanity.com/paper/2003.02944/) (2020)
- [Learning Attributed Graph Representations with Communicative Message Passing Transformer](https://mlsanity.com/paper/2107.08773/) (2021)
- [Learning to SMILE(S)](https://mlsanity.com/paper/1602.06289/) (2016)
- [Knowledge-aware Contrastive Molecular Graph Learning](https://mlsanity.com/paper/2103.13047/) (2021)

## Built on by (most cited first, 30 of 50)

- [A Comprehensive Survey on Deep Graph Representation Learning](https://mlsanity.com/paper/2304.05055/) (2023)
- [GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning](https://mlsanity.com/paper/2306.13089/) (2023)
- [Graph-based Molecular Representation Learning](https://mlsanity.com/paper/2207.04869/) (2022)
- [MolFM: A Multimodal Molecular Foundation Model](https://mlsanity.com/paper/2307.09484/) (2023)
- [A Systematic Survey of Chemical Pre-trained Models](https://mlsanity.com/paper/2210.16484/) (2022)
- [Fractional Denoising for 3D Molecular Pre-training](https://mlsanity.com/paper/2307.10683/) (2023)
- [MEAformer: Multi-modal Entity Alignment Transformer for Meta Modality Hybrid](https://mlsanity.com/paper/2212.14454/) (2022)
- [A Survey of Pretraining on Graphs: Taxonomy, Methods, and Applications](https://mlsanity.com/paper/2202.07893/) (2022)
- [Graph Neural Networks for Molecules](https://mlsanity.com/paper/2209.05582/) (2022)
- [Rethinking Uncertainly Missing and Ambiguous Visual Modality in Multi-Modal Entity Alignment](https://mlsanity.com/paper/2307.16210/) (2023)
- [SpeGCL: Self-supervised Graph Spectrum Contrastive Learning without Positive Samples](https://mlsanity.com/paper/2410.10365/) (2024)
- [Learning Invariant Molecular Representation in Latent Discrete Space](https://mlsanity.com/paper/2310.14170/) (2023)
- [Sliced Denoising: A Physics-Informed Molecular Pre-Training Method](https://mlsanity.com/paper/2311.02124/) (2023)
- [Pre-training with Fractional Denoising to Enhance Molecular Property Prediction](https://mlsanity.com/paper/2407.11086/) (2024)
- [Supervised Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2209.00078/) (2022)
- [Attention-wise masked graph contrastive learning for predicting molecular property](https://mlsanity.com/paper/2206.08262/) (2022)
- [KRACL: Contrastive Learning with Graph Context Modeling for Sparse Knowledge Graph Completion](https://mlsanity.com/paper/2208.07622/) (2022)
- [Graph Sampling-based Meta-Learning for Molecular Property Prediction](https://mlsanity.com/paper/2306.16780/) (2023)
- [Deep reinforcement learning for process design: Review and perspective](https://mlsanity.com/paper/2308.07822/) (2023)
- [EvoluNet: Advancing Dynamic Non-IID Transfer Learning on Graphs](https://mlsanity.com/paper/2305.00664/) (2023)
- [ESM All-Atom: Multi-scale Protein Language Model for Unified Molecular Modeling](https://mlsanity.com/paper/2403.12995/) (2024)
- [Knowledge-augmented Graph Machine Learning for Drug Discovery: A Survey](https://mlsanity.com/paper/2302.08261/) (2023)
- [Bi-level Contrastive Learning for Knowledge-Enhanced Molecule Representations](https://mlsanity.com/paper/2306.01631/) (2023)
- [UniMAP: Universal SMILES-Graph Representation Learning](https://mlsanity.com/paper/2310.14216/) (2023)
- [Impact of Domain Knowledge and Multi-Modality on Intelligent Molecular Property Prediction: A Systematic Survey](https://mlsanity.com/paper/2402.07249/) (2024)
- [ENGAGE: Explanation Guided Data Augmentation for Graph Representation Learning](https://mlsanity.com/paper/2307.01053/) (2023)
- [MolKD: Distilling Cross-Modal Knowledge in Chemical Reactions for Molecular Property Prediction](https://mlsanity.com/paper/2305.01912/) (2023)
- [Atom-Motif Contrastive Transformer for Molecular Property Prediction](https://mlsanity.com/paper/2310.07351/) (2023)
- [MOL-Mamba: Enhancing Molecular Representation with Structural & Electronic Insights](https://mlsanity.com/paper/2412.16483/) (2024)
- [Knowledge-informed Molecular Learning: A Survey on Paradigm Transfer](https://mlsanity.com/paper/2202.10587/) (2022)

## Abstract (arXiv)

Molecular representation learning contributes to multiple downstream tasks such as molecular property prediction and drug design. To properly represent molecules, graph contrastive learning is a promising paradigm as it utilizes self-supervision signals and has no requirements for human annotations. However, prior works fail to incorporate fundamental domain knowledge into graph semantics and thus ignore the correlations between atoms that have common attributes but are not directly connected by bonds. To address these issues, we construct a Chemical Element Knowledge Graph (KG) to summarize microscopic associations between elements and propose a novel Knowledge-enhanced Contrastive Learning (KCL) framework for molecular representation learning. KCL framework consists of three modules. The first module, knowledge-guided graph augmentation, augments the original molecular graph based on the Chemical Element KG. The second module, knowledge-aware graph representation, extracts molecular representations with a common graph encoder for the original molecular graph and a Knowledge-aware Message Passing Neural Network (KMPNN) to encode complex information in the augmented molecular graph. The final module is a contrastive objective, where we maximize agreement between these two views of molecular graphs. Extensive experiments demonstrated that KCL obtained superior performances against state-of-the-art baselines on eight molecular datasets. Visualization experiments properly interpret what KCL has learned from atoms and attributes in the augmented molecular graphs. Our codes and data are available at https://github.com/ZJU-Fangyin/KCL.