# Learn molecular representations from large-scale unlabeled molecules for drug discovery

arXiv: https://arxiv.org/abs/2012.11175 · HTML: https://mlsanity.com/paper/2012.11175/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2020.

## Built on by (most cited first, 18 of 18)

- [ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction](https://mlsanity.com/paper/2106.06130/) (2021)
- [ChemBERTa-2: Towards Chemical Foundation Models](https://mlsanity.com/paper/2209.01712/) (2022)
- [Large AI Models in Health Informatics: Applications, Challenges, and the Future](https://mlsanity.com/paper/2303.11568/) (2023)
- [Fractional Denoising for 3D Molecular Pre-training](https://mlsanity.com/paper/2307.10683/) (2023)
- [Dual-view Molecule Pre-training](https://mlsanity.com/paper/2106.10234/) (2021)
- [Supervised Pretraining for Molecular Force Fields and Properties Prediction](https://mlsanity.com/paper/2211.14429/) (2022)
- [ESM All-Atom: Multi-scale Protein Language Model for Unified Molecular Modeling](https://mlsanity.com/paper/2403.12995/) (2024)
- [Improving Molecular Pretraining with Complementary Featurizations](https://mlsanity.com/paper/2209.15101/) (2022)
- [BatmanNet: Bi-branch Masked Graph Transformer Autoencoder for Molecular Representation](https://mlsanity.com/paper/2211.13979/) (2022)
- [MolKD: Distilling Cross-Modal Knowledge in Chemical Reactions for Molecular Property Prediction](https://mlsanity.com/paper/2305.01912/) (2023)
- [Knowledge-informed Molecular Learning: A Survey on Paradigm Transfer](https://mlsanity.com/paper/2202.10587/) (2022)
- [GeoRecon: Graph-Level Representation Learning for 3D Molecules via Reconstruction-Based Pretraining](https://mlsanity.com/paper/2506.13174/) (2025)
- [Pre-trained Molecular Language Models with Random Functional Group Masking](https://mlsanity.com/paper/2411.01401/) (2024)
- [Structure-Based Drug Design via 3D Molecular Generative Pre-training and Sampling](https://mlsanity.com/paper/2402.14315/) (2024)
- [CandidateDrug4Cancer: An Open Molecular Graph Learning Benchmark on Drug Discovery for Cancer](https://mlsanity.com/paper/2203.00836/) (2022)
- [Self-Supervised Learning for Graph-Structured Data in Healthcare Applications: A Comprehensive Review](https://mlsanity.com/paper/2412.05312/) (2024)
- [3D-GSRD: 3D Molecular Graph Auto-Encoder with Selective Re-mask Decoding](https://mlsanity.com/paper/2510.16780/) (2025)
- [BiScale-GTR: Fragment-Aware Graph Transformers for Multi-Scale Molecular Representation Learning](https://mlsanity.com/paper/2604.06336/) (2026)

## Abstract (arXiv)

How to produce expressive molecular representations is a fundamental challenge in AI-driven drug discovery. Graph neural network (GNN) has emerged as a powerful technique for modeling molecular data. However, previous supervised approaches usually suffer from the scarcity of labeled data and have poor generalization capability. Here, we proposed a novel Molecular Pre-training Graph-based deep learning framework, named MPG, that leans molecular representations from large-scale unlabeled molecules. In MPG, we proposed a powerful MolGNet model and an effective self-supervised strategy for pre-training the model at both the node and graph-level. After pre-training on 11 million unlabeled molecules, we revealed that MolGNet can capture valuable chemistry insights to produce interpretable representation. The pre-trained MolGNet can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of drug discovery tasks, including molecular properties prediction, drug-drug interaction, and drug-target interaction, involving 13 benchmark datasets. Our work demonstrates that MPG is promising to become a novel approach in the drug discovery pipeline.