mlsanityData to 14 Oct 2026

MolXPT: Wrapping Molecules with Text for Generative Pre-training

arXiv 2305.10688 ↗First posted 18 May 2023cs.CLcited by 81

Not on the map. Why?

Abstract (arXiv)

Generative pre-trained Transformer (GPT) has demonstrates its great success in natural language processing and related techniques have been adapted into molecular modeling. Considering that text is the most important record for scientific discovery, in this paper, we propose MolXPT, a unified language model of text and molecules pre-trained on SMILES (a sequence representation of molecules) wrapped by text. Briefly, we detect the molecule names in each sequence and replace them to the corresponding SMILES. In this way, the SMILES could leverage the information from surrounding text, and vice versa. The above wrapped sequences, text sequences from PubMed and SMILES sequences from PubChem are all fed into a language model for pre-training. Experimental results demonstrate that MolXPT outperforms strong baselines of molecular property prediction on MoleculeNet, performs comparably to the best model in text-molecule translation while using less than half of its parameters, and enables zero-shot molecular generation without finetuning.

Key facts. Cited by 81 papers in the map (data through 14 Oct 2026). Builds on 22 of its 48 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

22 of 48 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  6. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  7. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  8. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  9. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  10. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  11. 2022Galactica: A Large Language Model for Sciencecited by 744
  12. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
10 more
  1. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  2. 2021PPT: Pre-trained Prompt Tuning for Few-shot Learningcited by 232
  3. 2021ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Predictioncited by 204
  4. 2018Fr'echet ChemNet Distance: A metric for generative models for molecules in drug discoverycited by 194
  5. 2022Translation between Molecules and Natural Languagecited by 177
  6. 2021Motif-based Graph Self-Supervised Learning for Molecular Property Predictioncited by 159
  7. 2022A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Languagecited by 131
  8. 2017"Found in Translation": Predicting Outcomes of Complex Organic Chemistry Reactions using Neural Sequence-to-Sequence Modelscited by 55
  9. 2022BERN2: an advanced neural biomedical named entity recognition and normalization toolcited by 42
  10. 2022AdaPrompt: Adaptive Model Training for Prompt-based NLPcited by 30

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 9; 2024: 39; 2025: 20; 2026: 13.

81 papers.

Sort:

By area:

  1. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 229
  2. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 206
  3. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 192
  4. 2023What can Large Language Models do in chemistry? A comprehensive benchmark on eight taskscited by 185
  5. 2023BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associationscited by 100
  6. 2023InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discoverycited by 86
  7. 2024Scientific Large Language Models: A Survey on Biological & Chemical Domainscited by 80
  8. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 77
  9. 2024BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuningcited by 54
  10. 2024Developing ChemDFM as a large language foundation model for chemistrycited by 52
  11. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 37
  12. 2025Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoningcited by 34
38 more
  1. 2024ChemReasoner: Heuristic Search over a Large Language Model's Knowledge Space using Quantum-Chemical Feedbackcited by 33
  2. 2024Materials science in the era of large language models: a perspectivecited by 24
  3. 2024MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Predictioncited by 24
  4. 2024MoleculeQA: A Dataset to Evaluate Factual Accuracy in Molecular Comprehensioncited by 22
  5. 2024LLaMo: Large Language Model-based Molecular Graph Assistantcited by 22
  6. 2024L+M-24: Building a Dataset for Language + Molecules @ ACL 2024cited by 21
  7. 2024ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Trainingcited by 21
  8. 2024A quantitative analysis of knowledge-learning preferences in large language models in molecular sciencecited by 18
  9. 2024Large Language Models are In-Context Molecule Learnerscited by 18
  10. 2023Monte Carlo Thought Search: Large Language Model Querying for Complex Scientific Reasoning in Catalyst Designcited by 17
  11. 2024Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Surveycited by 17
  12. 2024HIGHT: Hierarchical Graph Tokenization for Molecule-Language Alignmentcited by 17
  13. 2024ChemDFM-X: Towards Large Multimodal Model for Chemistrycited by 16
  14. 2024From Words to Molecules: A Survey of Large Language Models in Chemistrycited by 14
  15. 2024TAGMol: Target-Aware Gradient-guided Molecule Generationcited by 12
  16. 2023From Artificially Real to Real: Leveraging Pseudo Data from Large Language Models for Low-Resource Molecule Discoverycited by 11
  17. 2024LDMol: A Text-to-Molecule Diffusion Model with Structurally Informative Latent Space Surpasses AR Modelscited by 11
  18. 2024Learning Multi-view Molecular Representations with Structured and Unstructured Knowledgecited by 11
  19. 2024Geometry Informed Tokenization of Molecules for Language Model Generationcited by 10
  20. 2024Professional Agents -- Evolving Large Language Models into Autonomous Experts with Human-Level Competenciescited by 9
  21. 20243D-MolT5: Leveraging Discrete Structural Information for Molecule-Text Modelingcited by 9
  22. 2025Reasoning-Enhanced Large Language Models for Molecular Property Predictioncited by 9
  23. 2026MolecularIQ: Characterizing Chemical Reasoning Capabilities Through Symbolic Verification on Molecular Graphscited by 8
  24. 2025A Survey of AI for Materials Science: Foundation Models, LLM Agents, Datasets, and Toolscited by 7
  25. 2025KnowMol: Advancing Molecular Large Language Models with Multi-Level Chemical Knowledgecited by 7
  26. 2024MolTailor: Tailoring Chemical Molecular Representation to Specific Tasks via Text Promptscited by 6
  27. 2025Mol-LLM: Multimodal Generalist Molecular LLM with Improved Graph Utilizationcited by 6
  28. 2023Examining User-Friendly and Open-Sourced Large GPT Models: A Survey on Language, Multimodal, and Scientific GPT Modelscited by 4
  29. 2024Bridging Text and Molecule: A Survey on Multimodal Frameworks for Moleculecited by 4
  30. 2025Improving Chemical Understanding of LLMs via SMILES Parsingcited by 4
  31. 2024ALMol: Aligned Language-Molecule Translation LLMs through Offline Preference Contrastive Optimisationcited by 3
  32. 2024Instruction-Based Molecular Graph Generation with Unified Text-Graph Diffusion Modelcited by 3
  33. 2024Y-Mol: A Multiscale Biomedical Knowledge-Guided Large Language Model for Drug Developmentcited by 3
  34. 2024Exploring Optimal Transport-Based Multi-Grained Alignments for Text-Molecule Retrievalcited by 3
  35. 2024SMI-Editor: Edit-based SMILES Language Model with Fragment-level Supervisioncited by 3
  36. 2024Property Enhanced Instruction Tuning for Multi-task Molecule Generation with Large Language Modelscited by 3
  37. 2025TRIDENT: Tri-Modal Molecular Representation Learning with Taxonomic Annotations and Local Correspondencecited by 3
  38. 2024Instruction Multi-Constraint Molecular Generation Using a Teacher-Student Large Language Modelcited by 2

50 most cited of 81 All 81 (JSON)