mlsanityData to 7 Oct 2026

MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter

arXiv 2310.12798 ↗First posted 19 Oct 2023cs.CL cs.MMcited by 128

Not on the map. Why?

Abstract (arXiv)

Language Models (LMs) have demonstrated impressive molecule understanding ability on various 1D text-related tasks. However, they inherently lack 2D graph perception - a critical ability of human professionals in comprehending molecules' topological structures. To bridge this gap, we propose MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter. MolCA enables an LM (e.g., Galactica) to understand both text- and graph-based molecular contents via the cross-modal projector. Specifically, the cross-modal projector is implemented as a Q-Former to connect a graph encoder's representation space and an LM's text space. Further, MolCA employs a uni-modal adapter (i.e., LoRA) for the LM's efficient adaptation to downstream tasks. Unlike previous studies that couple an LM with a graph encoder via cross-modal contrastive learning, MolCA retains the LM's ability of open-ended text generation and augments it with 2D graph information. To showcase its effectiveness, we extensively benchmark MolCA on tasks of molecule captioning, IUPAC name prediction, and molecule-text retrieval, on which MolCA significantly outperforms the baselines. Our codes and checkpoints can be found at https://github.com/acharkq/MolCA.

Key facts. Cited by 128 papers in the map (data through 7 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 19 Oct 2023.

Builds on

30 of 51 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2018How Powerful are Graph Neural Networks?cited by 4,355
  10. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  12. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
18 more
  1. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  2. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  3. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  4. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  5. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  6. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  7. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  8. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  9. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  10. 2022Galactica: A Large Language Model for Sciencecited by 736
  11. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  12. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  13. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  14. 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 165
  15. 2022Linearly Mapping from Image to Text Spacecited by 139
  16. 2022A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Languagecited by 129
  17. 2023Unifying Molecular and Textual Representations via Multi-task Language Modellingcited by 92
  18. 2023Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Languagecited by 44

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 51; 2025: 47; 2026: 25.

128 papers.

Sort:

By area:

  1. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  2. 2024LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Datasetcited by 124
  3. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  4. 2023InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discoverycited by 86
  5. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 84
  6. 2024A Survey of Large Language Models for Graphscited by 71
  7. 2024A Survey on LoRA of Large Language Modelscited by 67
  8. 2024Towards 3D Molecule-Text Interpretation in Language Modelscited by 65
  9. 2023Graph Prompt Learning: A Comprehensive Survey and Beyondcited by 61
  10. 2024BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuningcited by 54
  11. 2023Rethinking Tokenizer and Decoder in Masked Graph Modeling for Moleculescited by 49
  12. 2024How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehensioncited by 30
38 more
  1. 2024SciLitLLM: How to Adapt LLMs for Scientific Literature Understandingcited by 29
  2. 2024Multimodal Large Language Models for Inverse Molecular Design with Retrosynthetic Planningcited by 29
  3. 2024Heterogeneous Contrastive Learning for Foundation Models and Beyondcited by 27
  4. 2024Diffusion Models in textitDe Novo Drug Designcited by 26
  5. 2024ProtT3: Protein-to-Text Generation for Text-based Protein Understandingcited by 25
  6. 2024MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Predictioncited by 24
  7. 2024UniMoT: Unified Molecule-Text Language Model with Discrete Token Representationcited by 23
  8. 2024MolTC: Towards Molecular Relational Modeling In Language Modelscited by 22
  9. 2024LLaMo: Large Language Model-based Molecular Graph Assistantcited by 22
  10. 2024Speak-to-Structure: Evaluating LLMs in Open-domain Natural Language-Driven Molecule Generationcited by 22
  11. 2024L+M-24: Building a Dataset for Language + Molecules @ ACL 2024cited by 21
  12. 2024ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Trainingcited by 21
  13. 2024EXGC: Bridging Efficiency and Explainability in Graph Condensationcited by 20
  14. 2024MoleculeQA: A Dataset to Evaluate Factual Accuracy in Molecular Comprehensioncited by 19
  15. 2024ReactXT: Understanding Molecular "Reaction-ship" via Reaction-Contextualized Molecule-Text Pretrainingcited by 19
  16. 2024A quantitative analysis of knowledge-learning preferences in large language models in molecular sciencecited by 18
  17. 2024Large Language Models are In-Context Molecule Learnerscited by 18
  18. 2024HIGHT: Hierarchical Graph Tokenization for Molecule-Language Alignmentcited by 17
  19. 2024MolParser: End-to-end Visual Recognition of Molecule Structures in the Wildcited by 17
  20. 2025NExT-Mol: 3D Diffusion Meets 1D Language Modeling for 3D Molecule Generationcited by 17
  21. 2024Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Surveycited by 16
  22. 2024ChemDFM-X: Towards Large Multimodal Model for Chemistrycited by 16
  23. 2025Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Modelcited by 16
  24. 2024From Words to Molecules: A Survey of Large Language Models in Chemistrycited by 14
  25. 2024PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomescited by 14
  26. 2024Can Graph Neural Networks Learn Language with Extremely Weak Text Supervision?cited by 13
  27. 2024GraphTeam: Facilitating Large Language Model-based Graph Analysis via Multi-Agent Collaborationcited by 12
  28. 2025Graph2text or Graph2token: A Perspective of Large Language Models for Graph Learningcited by 12
  29. 2025A Survey of Graph Transformers: Architectures, Theories and Applicationscited by 12
  30. 2024LDMol: A Text-to-Molecule Diffusion Model with Structurally Informative Latent Space Surpasses AR Modelscited by 11
  31. 2024MolReFlect: Towards In-Context Fine-grained Alignments between Molecules and Textscited by 11
  32. 2024From Generalist to Specialist: A Survey of Large Language Models for Chemistrycited by 11
  33. 2025MolReasoner: Toward Effective and Interpretable Reasoning for Molecular LLMscited by 11
  34. 2025Large Language-Geometry Model: When LLM meets Equivariancecited by 10
  35. 20243D-MolT5: Leveraging Discrete Structural Information for Molecule-Text Modelingcited by 9
  36. 2024Advancing Molecular Graph-Text Pre-training via Fine-grained Alignmentcited by 9
  37. 20243M-Diffusion: Latent Multi-Modal Diffusion for Language-Guided Molecular Structure Generationcited by 8
  38. 2025OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discoverycited by 8

50 most cited of 128 All 128 (JSON)