Multi-view biomedical foundation models for molecule-target and property prediction
Not on the map. Why?
Abstract (arXiv)
Quality molecular representations are key to foundation model development in bio-medical research. Previous efforts have typically focused on a single representation or molecular view, which may have strengths or weaknesses on a given task. We develop Multi-view Molecular Embedding with Late Fusion (MMELON), an approach that integrates graph, image and text views in a foundation model setting and may be readily extended to additional representations. Single-view foundation models are each pre-trained on a dataset of up to 200M molecules. The multi-view model performs robustly, matching the performance of the highest-ranked single-view. It is validated on over 120 tasks, including molecular solubility, ADME properties, and activity against G Protein-Coupled receptors (GPCRs). We identify 33 GPCRs that are related to Alzheimer's disease and employ the multi-view model to select strong binders from a compound screen. Predictions are validated through structure-based modeling and identification of key binding motifs.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 29 of its 100 references that are in the map. First posted on arXiv 25 Oct 2024.
Builds on
29 of 100 references.
References · 29
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2023A Survey of Large Language Modelscited by 2,537
- 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
- 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
- 2019Strategies for Pre-training Graph Neural Networkscited by 964
- 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
- 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
- 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 340
- 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 332
17 more
- 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
- 2021Large-Scale Chemical Language Representations Capture Molecular Structure and Propertiescited by 203
- 2022Pure Transformers are Powerful Graph Learnerscited by 188
- 2022ChemBERTa-2: Towards Chemical Foundation Modelscited by 123
- 2021Geometric Deep Learning on Molecular Representationscited by 113
- 2022Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networkscited by 92
- 2016Low Data Drug Discovery with One-shot Learningcited by 91
- 2023Attending to Graph Transformerscited by 81
- 2017"Found in Translation": Predicting Outcomes of Complex Organic Chemistry Reactions using Neural Sequence-to-Sequence Modelscited by 54
- 2024Multimodal Fusion on Low-quality Data: A Comprehensive Surveycited by 48
- 2022Unified 2D and 3D Pre-Training of Molecular Representationscited by 45
- 2022FP-GNN: a versatile deep learning architecture for enhanced molecular property predictioncited by 31
- 2017Chemception: A Deep Neural Network with Minimal Chemistry Knowledge Matches the Performance of Expert-developed QSAR/QSPR Modelscited by 18
- 2022BARTSmiles: Generative Masked Language Models for Molecular Representationscited by 18
- 2019Combining docking pose rank and structure with deep learning improves protein-ligand binding mode predictioncited by 10
- 2022Improving Molecular Pretraining with Complementary Featurizationscited by 4
- 2023Integrating Chemical Language and Molecular Graph in Multimodal Fused Deep Learning for Drug Property Predictioncited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.