# Mol-JEPA: A multimodal Joint Embedding Predictive Architecture for Molecules

arXiv: https://arxiv.org/abs/2608.22642 · HTML: https://mlsanity.com/paper/2608.22642/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 80 references that are in the map.
- First posted on arXiv 23 Aug 2026.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning](https://mlsanity.com/paper/2203.02053/) (2022)
- [Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification](https://mlsanity.com/paper/2009.03509/) (2020)
- [Understanding Dimensional Collapse in Contrastive Self-supervised Learning](https://mlsanity.com/paper/2110.09348/) (2021)
- [ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction](https://mlsanity.com/paper/2010.09885/) (2020)
- [TabICL: A Tabular Foundation Model for In-Context Learning on Large Data](https://mlsanity.com/paper/2502.05564/) (2025)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics](https://mlsanity.com/paper/2511.08544/) (2025)
- [ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction](https://mlsanity.com/paper/2106.06130/) (2021)
- [Graph neural networks for materials science and chemistry](https://mlsanity.com/paper/2208.09481/) (2022)
- [UMA: A Family of Universal Models for Atoms](https://mlsanity.com/paper/2506.23971/) (2025)
- [Does GNN Pretraining Help Molecular Representation?](https://mlsanity.com/paper/2207.06010/) (2022)
- [MolFM: A Multimodal Molecular Foundation Model](https://mlsanity.com/paper/2307.09484/) (2023)
- [Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language](https://mlsanity.com/paper/2303.03363/) (2023)
- [Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets](https://mlsanity.com/paper/2310.04292/) (2023)
- [Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learning](https://mlsanity.com/paper/2505.12477/) (2025)
- [GraphCL: Contrastive Self-Supervised Learning of Graph Representations](https://mlsanity.com/paper/2007.08025/) (2020)
- [Benchmarking Pretrained Molecular Embedding Models For Molecular Representation Learning](https://mlsanity.com/paper/2508.06199/) (2025)
- [Deep Learning Foundation Models from Classical Molecular Descriptors](https://mlsanity.com/paper/2506.15792/) (2025)
- [stable-pretraining-v1: Foundation Model Research Made Simple](https://mlsanity.com/paper/2511.19484/) (2025)
- [Exploring QSAR Models for Activity-Cliff Prediction](https://mlsanity.com/paper/2301.13644/) (2023)
- [WelQrate: Defining the Gold Standard in Small Molecule Drug Discovery Benchmarking](https://mlsanity.com/paper/2411.09820/) (2024)
- [MolMix: A Simple Yet Effective Baseline for Multimodal Molecular Representation Learning](https://mlsanity.com/paper/2410.07981/) (2024)
- [Do Larger Models Really Win in Drug Discovery? A Benchmark Assessment of Model Scaling in AI-Driven Molecular Property and Activity Prediction](https://mlsanity.com/paper/2604.26498/) (2026)
- [Joint Embedding Predictive Architecture for self-supervised pretraining on polymer molecular graphs](https://mlsanity.com/paper/2506.18194/) (2025)
- [When to Align, When to Predict: A Phase Diagram for Multimodal Learning](https://mlsanity.com/paper/2606.11190/) (2026)

## Built on by (most cited first, 2 of 2)

- [Can phenotypic activity be predicted without experimental readouts?](https://mlsanity.com/paper/2610.07997/) (2026)
- [Atom-JEPA: Joint-Embedding Predictive Architecture for 3D Atomistic Systems](https://mlsanity.com/paper/2610.08400/) (2026)

## Abstract (arXiv)

Despite recent advances in molecular foundation models, several limitations remain, such as chemically invalid augmentations, modality collapse, and incomplete representation of biochemical environments. To address these challenges, we present Mol-JEPA, a scalable framework for learning molecular world models. Rather than relying on suboptimal molecular perturbations, our model uses modality masking to exploit information from molecular structures, cellular phenotypes, binding affinities, ADMET profiles, quantum chemistry simulations and other drug discovery data. Across various benchmarks, we show that the representations learned by Mol-JEPA deliver strong performance, demonstrating the value of incorporating biochemical context through latent space prediction.