# MV-CLAM: Multi-View Molecular Interpretation with Cross-Modal Projection via Language Model

arXiv: https://arxiv.org/abs/2503.04780 · HTML: https://mlsanity.com/paper/2503.04780/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 46 references that are in the map.
- First posted on arXiv 23 Feb 2025.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [Analyzing Learned Molecular Representations for Property Prediction](https://mlsanity.com/paper/1904.01561/) (2019)
- [Pre-training Molecular Graph Representation with 3D Geometry](https://mlsanity.com/paper/2110.07728/) (2021)
- [ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction](https://mlsanity.com/paper/2106.06130/) (2021)
- [Translation between Molecules and Natural Language](https://mlsanity.com/paper/2204.11817/) (2022)
- [Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing](https://mlsanity.com/paper/2212.10789/) (2022)
- [Linearly Mapping from Image to Text Space](https://mlsanity.com/paper/2209.15162/) (2022)
- [A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language](https://mlsanity.com/paper/2209.05481/) (2022)
- [MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter](https://mlsanity.com/paper/2310.12798/) (2023)
- [Molecule Attention Transformer](https://mlsanity.com/paper/2002.08264/) (2020)
- [InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery](https://mlsanity.com/paper/2311.16208/) (2023)
- [GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text](https://mlsanity.com/paper/2308.06911/) (2023)
- [Fine-Grained Semantically Aligned Vision-Language Pre-Training](https://mlsanity.com/paper/2208.02515/) (2022)
- [Towards 3D Molecule-Text Interpretation in Language Models](https://mlsanity.com/paper/2401.13923/) (2024)
- [Graph-based Molecular Representation Learning](https://mlsanity.com/paper/2207.04869/) (2022)
- [Molformer: Motif-based Transformer on 3D Heterogeneous Molecular Graphs](https://mlsanity.com/paper/2110.01191/) (2021)
- [Modeling Caption Diversity in Contrastive Vision-Language Pretraining](https://mlsanity.com/paper/2405.00740/) (2024)
- [UniMoT: Unified Molecule-Text Language Model with Discrete Token Representation](https://mlsanity.com/paper/2408.00863/) (2024)
- [Diffusion-based Generative AI for Exploring Transition States from 2D Molecular Graphs](https://mlsanity.com/paper/2304.12233/) (2023)
- [Learning Multi-view Molecular Representations with Structured and Unstructured Knowledge](https://mlsanity.com/paper/2406.09841/) (2024)

## Built on by (most cited first, 2 of 2)

- [mCLM: A Modular Chemical Language Model that Generates Functional and Makeable Molecules](https://mlsanity.com/paper/2505.12565/) (2025)
- [Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?](https://mlsanity.com/paper/2506.10912/) (2025)

## Abstract (arXiv)

Human expertise in chemistry and biomedicine relies on contextual molecular understanding, a capability that large language models (LLMs) can extend through fine-grained alignment between molecular structures and text. Recent multimodal learning advances focus on cross-modal alignment, but existing molecule-text models ignore complementary information in different molecular views and rely on single-view representations, limiting molecular understanding. Moreover, na"ive multi-view alignment strategies face two challenges: (1) separate aligned spaces with inconsistent mappings between molecule and text embeddings, and that (2) existing loss objectives fail to preserve complementary information for fine-grained alignment. This can limit the LLM's ability to fully understand the molecular properties. To address these issues, we propose MV-CLAM, a novel framework that aligns multi-view molecular representations into a unified textual space using a multi-query transformer (MQ-Former). Our approach ensures cross-view consistency while a token-level contrastive loss preserves diverse molecular features across textual queries. MV-CLAM enhances molecular reasoning, improving retrieval and captioning accuracy. The source code of MV-CLAM is available in https://github.com/sumin124/mv-clam.git.