mlsanityData to 14 Oct 2026

GLACIER: A Multimodal Student-Teacher Foundation Model for Molecular Property Prediction

arXiv 2606.11382 ↗First posted 9 Jun 2026cs.LG q-bio.BMcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep learning models facilitate the discovery of molecules with tailored properties among billions of candidate compounds. However, the computational burden to develop and deploy state-of-the-art models continuously increases, limiting their scalability. Most large-scale models are unimodal in nature and overlook the potential to leverage complementary molecular data modalities. To address these shortcomings, this paper introduces the Graph-Language Alignment for Chemical Inference and Exploration using Representations (GLACIER) model, a student-teacher framework that integrates molecular graphs, SMILES strings, and physicochemical descriptors to learn rich molecular embeddings. Our framework consists of three stages: (1) we pretrain three student encoders on 100,000 drug-like molecules: a message-passing neural network for molecular graphs, a transformer-based encoder for SMILES strings, and a multilayer perceptron for physicochemical descriptors, (2) we fuse these student modalities using a novel Finsler geometry-aware module, and (3) distill complementary knowledge from large teacher models, including MiniMol and MolFormer, into a single lightweight model via contrastive learning. We demonstrate that GLACIER is a robust framework that delivers high predictive performance and computational efficiency in complex molecular property prediction tasks. Our code is publicly available at https://github.com/eemokey/glacier.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 60 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

26 of 60 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  6. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  7. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  8. 2019Analyzing Learned Molecular Representations for Property Predictioncited by 375
  9. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 337
  10. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 283
  11. 2021Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Developmentcited by 264
  12. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
14 more
  1. 2021Large-Scale Chemical Language Representations Capture Molecular Structure and Propertiescited by 206
  2. 2022On The Computational Complexity of Self-Attentioncited by 139
  3. 2023GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Textcited by 82
  4. 2017SMILES Enumeration as Data Augmentation for Neural Network Modeling of Moleculescited by 59
  5. 2021Relative Molecule Self-Attention Transformercited by 21
  6. 2023Fragment-based Pretraining and Finetuning on Molecular Graphscited by 20
  7. 2024textttMiniMol: A Parameter-Efficient Foundation Model for Molecular Learningcited by 20
  8. 2025Benchmarking Pretrained Molecular Embedding Models For Molecular Representation Learningcited by 15
  9. 2023Accelerating Molecular Graph Neural Networks via Knowledge Distillationcited by 11
  10. 2024ChemFM as a Scaling Law Guided Foundation Model Pre-trained on Informative Chemicalscited by 10
  11. 2024Advancing Molecular Graph-Text Pre-training via Fine-grained Alignmentcited by 9
  12. 2025Finsler Multi-Dimensional Scaling: Manifold Learning for Asymmetric Dimensionality Reduction and Embeddingcited by 8
  13. 2025TRIDENT: Tri-Modal Molecular Representation Learning with Taxonomic Annotations and Local Correspondencecited by 3
  14. 2025CL-MFAP: A Contrastive Learning-Based Multimodal Foundation Model for Molecular Property Prediction and Antibiotic Screeningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Thinking like a CHEMIST: Combined Heterogeneous Embedding Model Integrating Structure and Tokens
  2. 2025M2LLM: Multi-view Molecular Representation Learning with Large Language Models
  3. 2025Reasoning-Enhanced Large Language Models for Molecular Property Prediction
  4. 2024Explainable Molecular Property Prediction: Aligning Chemical Concepts with Predictions via Language Models
  5. 2026MolE-RAG: Molecular Structure-Enhanced Retrieval-Augmented Generation for Chemistry
  6. 2024LLM and GNN are Complementary: Distilling LLM for Multimodal Graph Learning
  7. 2026MolEmb: Multimodal Large Language Models Can Be Strong Molecular Embedding Models
  8. 2025Enhancing Molecular Property Prediction with Knowledge from Large Language Models