# Probabilistic Contrastive Pretraining for Multi-task ADME Property Prediction

arXiv: https://arxiv.org/abs/2606.11508 · HTML: https://mlsanity.com/paper/2606.11508/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 55 references that are in the map.
- First posted on arXiv 9 Jun 2026.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization](https://mlsanity.com/paper/1908.01000/) (2019)
- [Analyzing Learned Molecular Representations for Property Prediction](https://mlsanity.com/paper/1904.01561/) (2019)
- [Which Tasks Should Be Learned Together in Multi-task Learning?](https://mlsanity.com/paper/1905.07553/) (2019)
- [ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction](https://mlsanity.com/paper/2010.09885/) (2020)
- [Large-Scale Chemical Language Representations Capture Molecular Structure and Properties](https://mlsanity.com/paper/2106.09553/) (2021)
- [SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery](https://mlsanity.com/paper/1911.04738/) (2019)
- [On the Scalability of GNNs for Molecular Graphs](https://mlsanity.com/paper/2404.11568/) (2024)
- [KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Prediction](https://mlsanity.com/paper/2206.03364/) (2022)
- [GPS++: An Optimised Hybrid MPNN/Transformer for Molecular Property Prediction](https://mlsanity.com/paper/2212.02229/) (2022)
- [Multi-task Neural Networks for QSAR Predictions](https://mlsanity.com/paper/1406.1231/) (2014)
- [Improving Small Molecule Generation using Mutual Information Machine](https://mlsanity.com/paper/2208.09016/) (2022)
- [Multi-channel learning for integrating structural hierarchies into context-dependent molecular representation](https://mlsanity.com/paper/2311.02798/) (2023)
- [SentenceMIM: A Latent Variable Language Model](https://mlsanity.com/paper/2003.02645/) (2020)
- [Multitask finetuning and acceleration of chemical pretrained models for small molecule drug property prediction](https://mlsanity.com/paper/2510.12719/) (2025)
- [High Mutual Information in Representation Learning with Symmetric Variational Inference](https://mlsanity.com/paper/1910.04153/) (2019)
- [Contrastive Mutual Information Learning: Toward Robust Representations without Positive-Pair Augmentations](https://mlsanity.com/paper/2509.21511/) (2025)

## Built on by (most cited first, 2 of 2)

- [AssayRouter: Historical Utility Priors for Frozen Molecular Predictor Routing](https://mlsanity.com/paper/2609.37285/) (2026)
- [Atom-JEPA: Joint-Embedding Predictive Architecture for 3D Atomistic Systems](https://mlsanity.com/paper/2610.08400/) (2026)

## Abstract (arXiv)

Accurate prediction of absorption, distribution, metabolism, and excretion (ADME) properties is critical to drug discovery, but remains challenging because ADME endpoints are noisy, interdependent, and often data-limited. We propose a molecular graph-transformer pretraining framework that combines chemistry-specific self-supervision with contrastive mutual information machine learning (cMIM). Our method encodes molecular graphs into latent variables, reconstructs SMILES strings from the graph-derived latent codes, and augments the contrastive objective with domain-specific self-supervised chemistry tasks. Rather than treating these tasks as auxiliary regularizers with separately tuned loss weights, we formulate reconstruction, contrastive discrimination, and chemistry-specific supervision as unit-weighted log-probability factors in a single probabilistic latent-variable objective. For fine-tuning, we propose a multi-task GNN readout architecture with task-specific multilayer perceptron heads, preserving shared representation learning while mitigating negative transfer and improving the modeling of heterogeneous, nonlinear task relationships. Across Biogen, ExpansionRX, and ChEMBL-MT, the resulting Contrastive KERMT pretraining improves over the KERMT baseline by 7.6%, 9.9%, and 9.5% respectively (averaged over significantly-improved endpoints). Adding ADME-adjacent molecules to the pretraining corpus further improves transfer, and the contrastive component sharpens chemically meaningful latent neighborhoods.