# Deep Learning Foundation Models from Classical Molecular Descriptors

arXiv: https://arxiv.org/abs/2506.15792 · HTML: https://mlsanity.com/paper/2506.15792/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 53 references that are in the map.
- First posted on arXiv 18 Jun 2025.

## Builds on (most cited first, 14 of 14)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Analyzing Learned Molecular Representations for Property Prediction](https://mlsanity.com/paper/1904.01561/) (2019)
- [ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction](https://mlsanity.com/paper/2010.09885/) (2020)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [Large-Scale Chemical Language Representations Capture Molecular Structure and Properties](https://mlsanity.com/paper/2106.09553/) (2021)
- [Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets](https://mlsanity.com/paper/2310.04292/) (2023)
- [On the Scalability of GNNs for Molecular Graphs](https://mlsanity.com/paper/2404.11568/) (2024)
- [Multitask Learning On Graph Neural Networks Applied To Molecular Property Predictions](https://mlsanity.com/paper/1910.13124/) (2019)
- [Generalizable, Fast, and Accurate DeepQSPR with fastprop](https://mlsanity.com/paper/2404.02058/) (2024)
- [Why Deep Models Often cannot Beat Non-deep Counterparts on Molecular Property Prediction?](https://mlsanity.com/paper/2306.17702/) (2023)
- [Pretraining Graph Transformers with Atom-in-a-Molecule Quantum Properties for Improved ADMET Modeling](https://mlsanity.com/paper/2410.08024/) (2024)

## Built on by (most cited first, 14 of 14)

- [Tabular foundation models for in-context prediction of molecular properties](https://mlsanity.com/paper/2604.16123/) (2026)
- [AIBuildAI-2: A Knowledge-Enhanced Agent for Automatically Building AI Models](https://mlsanity.com/paper/2605.27873/) (2026)
- [Mol-JEPA: A multimodal Joint Embedding Predictive Architecture for Molecules](https://mlsanity.com/paper/2608.22642/) (2026)
- [Advancing Ligand-based Virtual Screening and Molecular Generation with Pretrained Molecular Embedding Distance](https://mlsanity.com/paper/2604.24474/) (2026)
- [DrugSAGE:Self-evolving Agent Experience for Efficient State-of-the-Art Drug Discovery](https://mlsanity.com/paper/2605.15461/) (2026)
- [On Improving Graph Neural Networks for QSAR by Pre-training on Extended-Connectivity Fingerprints](https://mlsanity.com/paper/2605.10722/) (2026)
- [The Geometry of Activity Cliffs: Representation Dependence and Multi-Scale Characterization of Activity Landscapes](https://mlsanity.com/paper/2605.30831/) (2026)
- [LatentFlow: Visual Analytics for Latent Space Analysis in Molecular Graph Neural Networks](https://mlsanity.com/paper/2607.21941/) (2026)
- [Scikit-fingerprints: Python library for scikit-learn compatible molecular fingerprints and chemoinformatics](https://mlsanity.com/paper/2608.02027/) (2026)
- [CheMLFlow: An Open-Source Platform for Cheminformatics and Materials Informatics Applications](https://mlsanity.com/paper/2608.04942/) (2026)
- [Feature Selection for Tabular Foundation Models: Exploring Feature Utility in Low-Data Molecular Prediction](https://mlsanity.com/paper/2609.30789/) (2026)
- [AssayRouter: Historical Utility Priors for Frozen Molecular Predictor Routing](https://mlsanity.com/paper/2609.37285/) (2026)
- [Molecular Property Prediction under Structural Shift with Tabular Foundation Models](https://mlsanity.com/paper/2609.38744/) (2026)
- [From Benchmark to Bench: Can Agents Survive Real-World Drug Discovery?](https://mlsanity.com/paper/2610.06411/) (2026)

## Abstract (arXiv)

Fast and accurate data-driven prediction of molecular properties is pivotal to scientific advancements across myriad chemical domains. Deep learning methods have recently garnered much attention, despite their inability to outperform classical machine learning methods when tested on practical, real-world benchmarks with limited training data. This study seeks to bridge this gap with CheMeleon, a O(10M) parameter foundation model that enables directed message-passing neural networks to finally exceed the performance of classical methods. Evaluated on 58 benchmark datasets from Polaris and MoleculeACE, CheMeleon achieves a win rate of 75% on Polaris tasks, outperforming baselines like Random Forest (68%), fastprop (36%), and Chemprop (32%), and a 97% win rate on MoleculeACE assays, surpassing Random Forest (50%) and other foundation models. Unlike conventional pre-training approaches that rely on noisy experimental data or biased quantum mechanical simulations, CheMeleon utilizes low-noise molecular descriptors to learn rich and highly transferable molecular representations, suggesting a new avenue for foundation model pre-training.