mlsanityData to 7 Oct 2026

BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations

arXiv 2310.07276 ↗First posted 11 Oct 2023cs.CL cs.AI cs.LGcited by 100

Not on the map. Why?

Abstract (arXiv)

Recent advancements in biological research leverage the integration of molecules, proteins, and natural language to enhance drug discovery. However, current models exhibit several limitations, such as the generation of invalid molecular SMILES, underutilization of contextual information, and equal treatment of structured and unstructured knowledge. To address these issues, we propose BioT5, a comprehensive pre-training framework that enriches cross-modal integration in biology with chemical knowledge and natural language associations. BioT5 utilizes SELFIES for 100% robust molecular representations and extracts knowledge from the surrounding context of bio-entities in unstructured biological literature. Furthermore, BioT5 distinguishes between structured and unstructured knowledge, leading to more effective utilization of information. After fine-tuning, BioT5 shows superior performance across a wide range of tasks, demonstrating its strong capability of capturing underlying relations and properties of bio-entities. Our code is available at hrefhttps://github.com/QizhiPei/BioT5Github.

Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). Builds on 33 of its 101 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

33 of 101 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  9. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  10. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  11. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  12. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
21 more
  1. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  2. 2022Galactica: A Large Language Model for Sciencecited by 736
  3. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  4. 2015An Introduction to Convolutional Neural Networkscited by 510
  5. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 332
  6. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  7. 2021ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Predictioncited by 204
  8. 2018Fr'echet ChemNet Distance: A metric for generative models for molecules in drug discoverycited by 191
  9. 2022Translation between Molecules and Natural Languagecited by 173
  10. 2021Motif-based Graph Self-Supervised Learning for Molecular Property Predictioncited by 159
  11. 2022A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Languagecited by 129
  12. 2023Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspectivecited by 91
  13. 2023MolXPT: Wrapping Molecules with Text for Generative Pre-trainingcited by 81
  14. 2023ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Textscited by 78
  15. 2022PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understandingcited by 77
  16. 2022SELFIES and the future of molecular string representationscited by 76
  17. 2020MolTrans: Molecular Interaction Transformer for Drug Target Interaction Predictioncited by 63
  18. 2023A Text-guided Protein Design Frameworkcited by 61
  19. 2018DeepConv-DTI: Prediction of drug-target interactions via deep learning with convolution on protein sequencescited by 45
  20. 2022BERN2: an advanced neural biomedical named entity recognition and normalization toolcited by 42
  21. 2022Interpretable bilinear attention network with domain adaptation improves drug-target predictioncited by 39

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 44; 2025: 31; 2026: 25.

100 papers.

Sort:

By area:

  1. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  2. 2024Advancing Multimodal Medical Capabilities of Geminicited by 88
  3. 2024Scientific Large Language Models: A Survey on Biological & Chemical Domainscited by 80
  4. 2024Efficient Evolutionary Search Over Chemical Space with Large Language Modelscited by 59
  5. 2024BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuningcited by 54
  6. 2025TxGemma: Efficient and Agentic LLMs for Therapeuticscited by 40
  7. 2024LICO: Large Language Models for In-Context Molecular Optimizationcited by 26
  8. 2024MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Predictioncited by 24
  9. 2024Tx-LLM: A Large Language Model for Therapeuticscited by 23
  10. 2024MolTC: Towards Molecular Relational Modeling In Language Modelscited by 22
  11. 2024Speak-to-Structure: Evaluating LLMs in Open-domain Natural Language-Driven Molecule Generationcited by 22
  12. 2024L+M-24: Building a Dataset for Language + Molecules @ ACL 2024cited by 21
38 more
  1. 2024ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Trainingcited by 21
  2. 2024MoleculeQA: A Dataset to Evaluate Factual Accuracy in Molecular Comprehensioncited by 19
  3. 2024A quantitative analysis of knowledge-learning preferences in large language models in molecular sciencecited by 18
  4. 2024Large Language Models are In-Context Molecule Learnerscited by 18
  5. 2025Protein Large Language Models: A Comprehensive Surveycited by 18
  6. 2025RAG-Enhanced Collaborative LLM Agents for Drug Discoverycited by 18
  7. 2024Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Surveycited by 16
  8. 2025Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Modelcited by 16
  9. 2025ChemMLLM: Chemical Multimodal Large Language Modelcited by 16
  10. 2024From Words to Molecules: A Survey of Large Language Models in Chemistrycited by 14
  11. 2024PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomescited by 14
  12. 2024SmileyLlama: Modifying Large Language Models for Directed Chemical Space Explorationcited by 14
  13. 2024LDMol: A Text-to-Molecule Diffusion Model with Structurally Informative Latent Space Surpasses AR Modelscited by 11
  14. 2024Learning Multi-view Molecular Representations with Structured and Unstructured Knowledgecited by 11
  15. 2024MolReFlect: Towards In-Context Fine-grained Alignments between Molecules and Textscited by 11
  16. 2024Geometry Informed Tokenization of Molecules for Language Model Generationcited by 10
  17. 2024GP-GPT: Large Language Model for Gene-Phenotype Mappingcited by 10
  18. 2024Advancing biomolecular understanding and design following human instructionscited by 10
  19. 2025POLO: Preference-Guided Multi-Turn Reinforcement Learning for Lead Optimizationcited by 10
  20. 20243D-MolT5: Leveraging Discrete Structural Information for Molecule-Text Modelingcited by 9
  21. 2024Can LLMs Generate Diverse Molecules? Towards Alignment with Structural Diversitycited by 9
  22. 2024EvoLlama: Enhancing LLMs' Understanding of Proteins via Multimodal Structure and Sequence Representationscited by 9
  23. 2025Enhancing Chemical Reaction and Retrosynthesis Prediction with Large Language Model and Dual-task Learningcited by 9
  24. 2025SciReasoner: Laying the Scientific Reasoning Ground Across Disciplinescited by 9
  25. 2025OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discoverycited by 8
  26. 2025mCLM: A Modular Chemical Language Model that Generates Functional and Makeable Moleculescited by 8
  27. 2025Computational Protein Science in the Era of Large Language Models (LLMs)cited by 6
  28. 2025How to Detect and Defeat Molecular Mirage: A Metric-Driven Benchmark for Hallucination in LLM-based Molecular Comprehensioncited by 6
  29. 2024MolX: Enhancing Large Language Models for Molecular Understanding With A Multi-Modal Extensioncited by 5
  30. 2026Reference-guided Policy Optimization for Molecular Optimization via LLM Reasoningcited by 5
  31. 2024Bridging Text and Molecule: A Survey on Multimodal Frameworks for Moleculecited by 4
  32. 2024MAMMAL -- Molecular Aligned Multi-Modal Architecture and Languagecited by 4
  33. 2025Improving Chemical Understanding of LLMs via SMILES Parsingcited by 4
  34. 2025Rethinking Text-based Protein Understanding: Retrieval or LLM?cited by 4
  35. 2026Agentic reinforcement learning empowers next-generation chemical language models for molecular design and synthesiscited by 4
  36. 2024ChatCell: Facilitating Single-Cell Analysis with Natural Languagecited by 3
  37. 2024Data-Efficient Molecular Generation with Hierarchical Textual Inversioncited by 3
  38. 2024FusionDTI: Fine-grained Binding Discovery with Token-level Fusion for Drug-Target Interactioncited by 3

50 most cited of 100 All 100 (JSON)