mlsanityData to 7 Oct 2026

LLaMo: Large Language Model-based Molecular Graph Assistant

arXiv 2411.00871 ↗First posted 31 Oct 2024cs.LG cs.AI q-bio.MNcited by 22

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable generalization and instruction-following capabilities with instruction tuning. The advancements in LLMs and instruction tuning have led to the development of Large Vision-Language Models (LVLMs). However, the competency of the LLMs and instruction tuning have been less explored in the molecular domain. Thus, we propose LLaMo: Large Language Model-based Molecular graph assistant, which is an end-to-end trained large molecular graph-language model. To bridge the discrepancy between the language and graph modalities, we present the multi-level graph projector that transforms graph representations into graph tokens by abstracting the output representations of each GNN layer and motif representations with the cross-attention mechanism. We also introduce machine-generated molecular graph instruction data to instruction-tune the large molecular graph-language model for general-purpose molecule and language understanding. Our extensive experiments demonstrate that LLaMo shows the best performance on diverse tasks, such as molecular description generation, property prediction, and IUPAC name prediction. The code of LLaMo is available at https://github.com/mlvlab/LLaMo.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 47 of its 71 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

47 of 71 references.

References · 47

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2017Graph Attention Networkscited by 8,854
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2018How Powerful are Graph Neural Networks?cited by 4,355
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  10. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  11. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  12. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
35 more
  1. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  2. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  3. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  4. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  5. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  6. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
  7. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  8. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  9. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  10. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  11. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  12. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  13. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
  14. 2022Galactica: A Large Language Model for Sciencecited by 736
  15. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  16. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  17. 2018MolGAN: An implicit generative model for small molecular graphscited by 392
  18. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
  19. 2023Honeybee: Locality-enhanced Projector for Multimodal LLMcited by 195
  20. 2022Translation between Molecules and Natural Languagecited by 173
  21. 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 165
  22. 2021Motif-based Graph Self-Supervised Learning for Molecular Property Predictioncited by 159
  23. 2023Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Modelscited by 131
  24. 2022A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Languagecited by 129
  25. 2023MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adaptercited by 128
  26. 2021Geometric Deep Learning on Molecular Representationscited by 113
  27. 2023Unifying Molecular and Textual Representations via Multi-task Language Modellingcited by 92
  28. 2023InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discoverycited by 86
  29. 2023GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Textcited by 82
  30. 2023MolXPT: Wrapping Molecules with Text for Generative Pre-trainingcited by 81
  31. 2023GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learningcited by 70
  32. 2023MolFM: A Multimodal Molecular Foundation Modelcited by 50
  33. 2023TextBind: Multi-turn Interleaved Multimodal Instruction-following in the Wildcited by 22
  34. 2021Deformable Graph Convolutional Networkscited by 6
  35. 2023OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Modelscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 12.

22 papers.

Sort:

By area:

  1. 2025DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPOcited by 57
  2. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  3. 2025Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Modelcited by 16
  4. 2026Graph is a Substrate Across Data Modalitiescited by 13
  5. 2025Flex-Judge: Text-Only Reasoning Unleashes Zero-Shot Multimodal Evaluatorscited by 6
  6. 2025TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Taskscited by 5
  7. 2025Bidirectional Likelihood Estimation with Multi-Modal Large Language Models for Text-Video Retrievalcited by 4
  8. 2025TabFlash: Efficient Table Understanding with Progressive Question Conditioning and Token Focusingcited by 3
  9. 2025Captioning for Text-Video Retrieval via Dual-Group Direct Preference Optimizationcited by 1
  10. 2025MoRA: On-the-fly Molecule-aware Low-Rank Adaptation Framework for LLM-based Multi-Modal Molecular Assistantcited by 1
  11. 2026Entropy-Guided Dynamic Tokens for Graph-LLM Alignment in Molecular Understandingcited by 1
  12. 2026LLM-Guided Dynamic Action Spaces for Synthesizable Molecular Optimizationcited by 1
10 more
  1. 2025HSA-Net: Hierarchical and Structure-Aware Framework for Efficient and Scalable Molecular Language Modelingcited by 0
  2. 2026Improving Large Molecular Language Model via Relation-aware Multimodal Collaborationcited by 0
  3. 2026ES-Merging: Biological MLLM Merging via Embedding Space Signalscited by 0
  4. 2026MolDA: Molecular Understanding and Generation via Large Language Diffusion Modelcited by 0
  5. 2026C-MORAL: Controllable Multi-Objective Molecular Optimization with Reinforcement Alignment for LLMscited by 0
  6. 2026Bolek: A Multimodal Language Model for Molecular Reasoningcited by 0
  7. 2026MoleCode unlocks structural intelligence in large language modelscited by 0
  8. 2026Multi-Granular Rationale-Guided Molecular LLM for Property Predictioncited by 0
  9. 2026MolSC: Leveraging Substituent Contributions to Enhance Fine-grained Molecular Understanding in LLMscited by 0
  10. 2026SafeMol: Dual-Modality Safety Alignment for Molecular Multimodal Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Exploring Hierarchical Molecular Graph Representation in Multimodal LLMs
  2. 2025Mol-LLM: Multimodal Generalist Molecular LLM with Improved Graph Utilization
  3. 2024MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction
  4. 2024HIGHT: Hierarchical Graph Tokenization for Molecule-Language Alignment
  5. 2024Exploring the Potential of Large Language Models in Graph Generation
  6. 2025M2LLM: Multi-view Molecular Representation Learning with Large Language Models
  7. 2026Improving Molecular Property Prediction in Small Language Models Using Graph-based Tools
  8. 2024Multimodal Large Language Models for Inverse Molecular Design with Retrosynthetic Planning