mlsanityData to 14 Oct 2026

InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery

arXiv 2311.16208 ↗First posted 27 Nov 2023q-bio.BM cs.AI cs.LGcited by 86

Not on the map. Why?

Abstract (arXiv)

The rapid evolution of artificial intelligence in drug discovery encounters challenges with generalization and extensive training, yet Large Language Models (LLMs) offer promise in reshaping interactions with complex molecular data. Our novel contribution, InstructMol, a multi-modal LLM, effectively aligns molecular structures with natural language via an instruction-tuning approach, utilizing a two-stage training strategy that adeptly combines limited domain-specific data with molecular and textual information. InstructMol showcases substantial performance improvements in drug discovery-related molecular tasks, surpassing leading LLMs and significantly reducing the gap with specialized models, thereby establishing a robust foundation for a versatile and dependable drug discovery assistant.

Key facts. Cited by 86 papers in the map (data through 14 Oct 2026). Builds on 60 of its 100 references that are in the map. First posted on arXiv 27 Nov 2023.

Builds on

60 of 100 references.

References · 60

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  9. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  10. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  11. 2023Visual Instruction Tuningcited by 7,869
  12. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
38 more
  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  2. 2018How Powerful are Graph Neural Networks?cited by 4,393
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  8. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  9. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  10. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  11. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  12. 2023PaLM 2 Technical Reportcited by 1,162
  13. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  14. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  15. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  16. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  17. 2023A Survey on Multimodal Large Language Modelscited by 909
  18. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  19. 2022Galactica: A Large Language Model for Sciencecited by 744
  20. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  21. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
  22. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
  23. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 337
  24. 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
  25. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  26. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 248
  27. 2018Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Predictioncited by 229
  28. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 226
  29. 2022Translation between Molecules and Natural Languagecited by 177
  30. 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 166
  31. 20213D Infomax improves GNNs for Molecular Property Predictioncited by 146
  32. 2023Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Modelscited by 133
  33. 2022A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Languagecited by 131
  34. 2023X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languagescited by 128
  35. 2023MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adaptercited by 128
  36. 2022ChemBERTa-2: Towards Chemical Foundation Modelscited by 127
  37. 2023DetGPT: Detect What You Need via Reasoningcited by 112
  38. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 103

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 38; 2025: 34; 2026: 14.

86 papers.

Sort:

By area:

  1. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  2. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  3. 2024LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Datasetcited by 128
  4. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  5. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 77
  6. 2024A Survey of Large Language Models for Graphscited by 72
  7. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  8. 2024BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuningcited by 54
  9. 2024Developing ChemDFM as a large language foundation model for chemistrycited by 52
  10. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 37
  11. 2025Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoningcited by 34
  12. 2024ChemReasoner: Heuristic Search over a Large Language Model's Knowledge Space using Quantum-Chemical Feedbackcited by 33
38 more
  1. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  2. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  3. 2024How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehensioncited by 30
  4. 2024UniMoT: Unified Molecule-Text Language Model with Discrete Token Representationcited by 23
  5. 2024LLaMo: Large Language Model-based Molecular Graph Assistantcited by 22
  6. 2024L+M-24: Building a Dataset for Language + Molecules @ ACL 2024cited by 21
  7. 2024Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Surveycited by 17
  8. 2024HIGHT: Hierarchical Graph Tokenization for Molecule-Language Alignmentcited by 17
  9. 2025Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Modelcited by 17
  10. 2025ChemMLLM: Chemical Multimodal Large Language Modelcited by 17
  11. 2024PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomescited by 14
  12. 2025Graph2text or Graph2token: A Perspective of Large Language Models for Graph Learningcited by 12
  13. 2024Learning on Graphs with Large Language Models(LLMs): A Deep Dive into Model Robustnesscited by 11
  14. 2024Regression with Large Language Models for Materials and Molecular Property Predictioncited by 11
  15. 2024Structural Reasoning Improves Molecular Understanding of LLMcited by 11
  16. 2024From Generalist to Specialist: A Survey of Large Language Models for Chemistrycited by 11
  17. 2024Geometry Informed Tokenization of Molecules for Language Model Generationcited by 10
  18. 2024Advancing biomolecular understanding and design following human instructionscited by 10
  19. 20243D-MolT5: Leveraging Discrete Structural Information for Molecule-Text Modelingcited by 9
  20. 2025Enhancing Chemical Reaction and Retrosynthesis Prediction with Large Language Model and Dual-task Learningcited by 9
  21. 2025MolLangBench: A Comprehensive Benchmark for Language-Prompted Molecular Structure Recognition, Editing, and Generationcited by 9
  22. 2025Language-Enhanced Representation Learning for Single-Cell Transcriptomicscited by 7
  23. 2025A Comprehensive Data-centric Overview of Federated Graph Learningcited by 7
  24. 2025KnowMol: Advancing Molecular Large Language Models with Multi-Level Chemical Knowledgecited by 7
  25. 2025Mol-LLM: Multimodal Generalist Molecular LLM with Improved Graph Utilizationcited by 6
  26. 2025How to Detect and Defeat Molecular Mirage: A Metric-Driven Benchmark for Hallucination in LLM-based Molecular Comprehensioncited by 6
  27. 2025Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applicationscited by 6
  28. 2025Bridging Molecular Graphs and Large Language Modelscited by 5
  29. 2024Bridging Text and Molecule: A Survey on Multimodal Frameworks for Moleculecited by 4
  30. 2024Bioinformatics and Biomedical Informatics with ChatGPT: Year One Reviewcited by 4
  31. 2024SubGDiff: A Subgraph Diffusion Model to Improve Molecular Representation Learningcited by 4
  32. 2025Improving Chemical Understanding of LLMs via SMILES Parsingcited by 4
  33. 2025Rethinking Text-based Protein Understanding: Retrieval or LLM?cited by 4
  34. 2026SIQA: Toward Reliable Scientific Image Quality Assessmentcited by 4
  35. 2024CataLM: Empowering Catalyst Design Through Large Language Modelscited by 3
  36. 2024DRAK: Unlocking Molecular Insights with Domain-Specific Retrieval-Augmented Knowledge in LLMscited by 3
  37. 2024Instruction-Based Molecular Graph Generation with Unified Text-Graph Diffusion Modelcited by 3
  38. 2024The Graph's Apprentice: Teaching an LLM Low Level Knowledge for Circuit Quality Estimationcited by 3

50 most cited of 86 All 86 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025ChatMol: A Versatile Molecule Designer Based on the Numerically Enhanced Large Language Model
  2. 2026Evaluating the Progression of Large Language Model Capabilities for Small-Molecule Drug Design
  3. 2025Large Language Model Agent for Modular Task Execution in Drug Discovery
  4. 2024SmileyLlama: Modifying Large Language Models for Directed Chemical Space Exploration
  5. 2025BioMedGPT-Mol: Multi-task Learning for Molecular Understanding and Generation
  6. 2026DrugR: Optimizing Molecular Drugs through LLM-based Explicit Reasoning
  7. 2025DrugPilot: LLM-based Parameterized Reasoning Agent for Drug Discovery
  8. 2024Y-Mol: A Multiscale Biomedical Knowledge-Guided Large Language Model for Drug Development