MV-CLAM: Multi-View Molecular Interpretation with Cross-Modal Projection via Language Model
Not on the map. Why?
Abstract (arXiv)
Human expertise in chemistry and biomedicine relies on contextual molecular understanding, a capability that large language models (LLMs) can extend through fine-grained alignment between molecular structures and text. Recent multimodal learning advances focus on cross-modal alignment, but existing molecule-text models ignore complementary information in different molecular views and rely on single-view representations, limiting molecular understanding. Moreover, na"ive multi-view alignment strategies face two challenges: (1) separate aligned spaces with inconsistent mappings between molecule and text embeddings, and that (2) existing loss objectives fail to preserve complementary information for fine-grained alignment. This can limit the LLM's ability to fully understand the molecular properties. To address these issues, we propose MV-CLAM, a novel framework that aligns multi-view molecular representations into a unified textual space using a multi-query transformer (MQ-Former). Our approach ensures cross-view consistency while a token-level contrastive loss preserves diverse molecular features across textual queries. MV-CLAM enhances molecular reasoning, improving retrieval and captioning accuracy. The source code of MV-CLAM is available in https://github.com/sumin124/mv-clam.git.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 46 references that are in the map. First posted on arXiv 23 Feb 2025.
Builds on
30 of 46 references.
References · 30
- 2017Attention Is All You Needcited by 59,932
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
- 2020Graph Contrastive Learning with Augmentationscited by 1,203
- 2019Strategies for Pre-training Graph Neural Networkscited by 964
- 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
- 2022Galactica: A Large Language Model for Sciencecited by 736
- 2019Analyzing Learned Molecular Representations for Property Predictioncited by 373
18 more
- 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
- 2021ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Predictioncited by 204
- 2022Translation between Molecules and Natural Languagecited by 173
- 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 165
- 2022Linearly Mapping from Image to Text Spacecited by 139
- 2022A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Languagecited by 129
- 2023MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adaptercited by 128
- 2020Molecule Attention Transformercited by 92
- 2023InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discoverycited by 86
- 2023GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Textcited by 82
- 2022Fine-Grained Semantically Aligned Vision-Language Pre-Trainingcited by 68
- 2024Towards 3D Molecule-Text Interpretation in Language Modelscited by 65
- 2022Graph-based Molecular Representation Learningcited by 51
- 2021Molformer: Motif-based Transformer on 3D Heterogeneous Molecular Graphscited by 44
- 2024Modeling Caption Diversity in Contrastive Vision-Language Pretrainingcited by 42
- 2024UniMoT: Unified Molecule-Text Language Model with Discrete Token Representationcited by 23
- 2023Diffusion-based Generative AI for Exploring Transition States from 2D Molecular Graphscited by 19
- 2024Learning Multi-view Molecular Representations with Structured and Unstructured Knowledgecited by 11
Built on by
Citing papers per year: 2025: 2.
2 papers.