mlsanityData to 7 Oct 2026

MolBind: Multimodal Alignment of Language, Molecules, and Proteins

arXiv 2403.08167 ↗First posted 13 Mar 2024cs.LG cs.CL q-bio.QMcited by 7

Not on the map. Why?

Abstract (arXiv)

Recent advancements in biology and chemistry have leveraged multi-modal learning, integrating molecules and their natural language descriptions to enhance drug discovery. However, current pre-training frameworks are limited to two modalities, and designing a unified network to process different modalities (e.g., natural language, 2D molecular graphs, 3D molecular conformations, and 3D proteins) remains challenging due to inherent gaps among them. In this work, we propose MolBind, a framework that trains encoders for multiple modalities through contrastive learning, mapping all modalities to a shared feature space for multi-modal semantic alignment. To facilitate effective pre-training of MolBind on multiple modalities, we also build and collect a high-quality dataset with four modalities, MolBind-M4, including graph-language, conformation-language, graph-conformation, and conformation-protein paired data. MolBind shows superior zero-shot learning performance across a wide range of tasks, demonstrating its strong capability of capturing the underlying semantics of multiple modalities.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 22 of its 43 references that are in the map. First posted on arXiv 13 Mar 2024.

Builds on

22 of 43 references.

References · 22

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  3. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  4. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  5. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  6. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  7. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 366
  8. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
  9. 2020GEOM: Energy-annotated molecular conformations for property prediction and molecular generationcited by 238
  10. 2022Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pocketscited by 182
  11. 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 165
  12. 2022A 3D Generative Model for Structure-Based Drug Designcited by 148
10 more
  1. 2022A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Languagecited by 129
  2. 2023MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adaptercited by 128
  3. 2023Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Datacited by 68
  4. 2024Towards 3D Molecule-Text Interpretation in Language Modelscited by 65
  5. 2021Rotation Invariant Graph Neural Networks using Spin Convolutionscited by 52
  6. 2021Molecule3D: A Benchmark for Predicting 3D Geometries from Molecular Graphscited by 31
  7. 2023DrugCLIP: Contrastive Protein-Molecule Representation Learning for Virtual Screeningcited by 28
  8. 2023Learning How to Propagate Messages in Graph Neural Networkscited by 23
  9. 2023Towards equilibrium molecular conformation generation with GFlowNetscited by 14
  10. 20233D Molecular Geometry Analysis with 2D Graphscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 3.

7 papers.