mlsanityData to 7 Oct 2026

BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuning

arXiv 2402.17810 ↗First posted 27 Feb 2024q-bio.QM cs.AI cs.CEcited by 54

Not on the map. Why?

Abstract (arXiv)

Recent research trends in computational biology have increasingly focused on integrating text and bio-entity modeling, especially in the context of molecules and proteins. However, previous efforts like BioT5 faced challenges in generalizing across diverse tasks and lacked a nuanced understanding of molecular structures, particularly in their textual representations (e.g., IUPAC). This paper introduces BioT5+, an extension of the BioT5 framework, tailored to enhance biological research and drug discovery. BioT5+ incorporates several novel features: integration of IUPAC names for molecular understanding, inclusion of extensive bio-text and molecule data from sources like bioRxiv and PubChem, the multi-task instruction tuning for generality across tasks, and a numerical tokenization technique for improved processing of numerical data. These enhancements allow BioT5+ to bridge the gap between molecular representations and their textual descriptions, providing a more holistic understanding of biological entities, and largely improving the grounded reasoning of bio-text and bio-sequences. The model is pre-trained and fine-tuned with a large number of experiments, including 3 types of problems (classification, regression, generation), 15 kinds of tasks, and 21 total benchmark datasets, demonstrating the remarkable performance and state-of-the-art results in most cases. BioT5+ stands out for its ability to capture intricate relationships in biological data, thereby contributing significantly to bioinformatics and computational biology. Our code is available at urlhttps://github.com/QizhiPei/BioT5.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 38 of its 89 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

38 of 89 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  8. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  9. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  10. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  11. 2022Galactica: A Large Language Model for Sciencecited by 736
  12. 2015An Introduction to Convolutional Neural Networkscited by 510
26 more
  1. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 332
  2. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  3. 2021ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Predictioncited by 204
  4. 2018Fr'echet ChemNet Distance: A metric for generative models for molecules in drug discoverycited by 191
  5. 2022Translation between Molecules and Natural Languagecited by 173
  6. 2021Motif-based Graph Self-Supervised Learning for Molecular Property Predictioncited by 159
  7. 2021Investigating the Limitations of Transformers with Simple Arithmetic Taskscited by 136
  8. 2022A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Languagecited by 129
  9. 2023MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adaptercited by 128
  10. 2023The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4cited by 112
  11. 2023BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associationscited by 100
  12. 2023BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicinecited by 99
  13. 2023Unifying Molecular and Textual Representations via Multi-task Language Modellingcited by 92
  14. 2023Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspectivecited by 91
  15. 2023InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discoverycited by 86
  16. 2023Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Taskscited by 84
  17. 2023GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Textcited by 82
  18. 2023MolXPT: Wrapping Molecules with Text for Generative Pre-trainingcited by 81
  19. 2022PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understandingcited by 77
  20. 2020MolTrans: Molecular Interaction Transformer for Drug Target Interaction Predictioncited by 63
  21. 2023A Text-guided Protein Design Frameworkcited by 61
  22. 2023MolFM: A Multimodal Molecular Foundation Modelcited by 50
  23. 2018DeepConv-DTI: Prediction of drug-target interactions via deep learning with convolution on protein sequencescited by 45
  24. 2022BERN2: an advanced neural biomedical named entity recognition and normalization toolcited by 42
  25. 2022Interpretable bilinear attention network with domain adaptation improves drug-target predictioncited by 39
  26. 2023Prot2Text: Multimodal Protein's Function Generation with GNNs and Transformerscited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 19; 2025: 24; 2026: 11.

54 papers.

Sort:

By area:

  1. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 96
  2. 2025Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Surveycited by 47
  3. 2025SciToolAgent: A Knowledge Graph-Driven Scientific Agent for Multi-Tool Integrationcited by 40
  4. 2025Protein Large Language Models: A Comprehensive Surveycited by 18
  5. 2024Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Surveycited by 16
  6. 2025ChemMLLM: Chemical Multimodal Large Language Modelcited by 16
  7. 2024PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomescited by 14
  8. 2024Causal Representation Learning from Multimodal Biomedical Observationscited by 12
  9. 2024LDMol: A Text-to-Molecule Diffusion Model with Structurally Informative Latent Space Surpasses AR Modelscited by 11
  10. 2024Learning Multi-view Molecular Representations with Structured and Unstructured Knowledgecited by 11
  11. 2025MolReasoner: Toward Effective and Interpretable Reasoning for Molecular LLMscited by 11
  12. 2024Regression with Large Language Models for Materials and Molecular Property Predictioncited by 10
38 more
  1. 2024GP-GPT: Large Language Model for Gene-Phenotype Mappingcited by 10
  2. 2024Advancing biomolecular understanding and design following human instructionscited by 10
  3. 20243D-MolT5: Leveraging Discrete Structural Information for Molecule-Text Modelingcited by 9
  4. 2024Can LLMs Generate Diverse Molecules? Towards Alignment with Structural Diversitycited by 9
  5. 2024EvoLlama: Enhancing LLMs' Understanding of Proteins via Multimodal Structure and Sequence Representationscited by 9
  6. 2025Enhancing Chemical Reaction and Retrosynthesis Prediction with Large Language Model and Dual-task Learningcited by 9
  7. 2025SciReasoner: Laying the Scientific Reasoning Ground Across Disciplinescited by 9
  8. 2025Reasoning-Enhanced Large Language Models for Molecular Property Predictioncited by 9
  9. 2025OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discoverycited by 8
  10. 2025mCLM: A Modular Chemical Language Model that Generates Functional and Makeable Moleculescited by 8
  11. 2025Steering Protein Language Modelscited by 7
  12. 2025Computational Protein Science in the Era of Large Language Models (LLMs)cited by 6
  13. 2025Mol-LLM: Multimodal Generalist Molecular LLM with Improved Graph Utilizationcited by 6
  14. 2024MolX: Enhancing Large Language Models for Molecular Understanding With A Multi-Modal Extensioncited by 5
  15. 2024Structure-Enhanced Protein Instruction Tuning: Towards General-Purpose Protein Understanding with LLMscited by 4
  16. 2024MAMMAL -- Molecular Aligned Multi-Modal Architecture and Languagecited by 4
  17. 2025Rethinking Text-based Protein Understanding: Retrieval or LLM?cited by 4
  18. 2024Instruction-Based Molecular Graph Generation with Unified Text-Graph Diffusion Modelcited by 3
  19. 2025TRIDENT: Tri-Modal Molecular Representation Learning with Taxonomic Annotations and Local Correspondencecited by 3
  20. 2024A Self-feedback Knowledge Elicitation Approach for Chemical Reaction Predictionscited by 2
  21. 2025AviationLLM: An LLM-based Knowledge System for Aviation Trainingcited by 2
  22. 2025Bridging the Gap Between Molecule and Textual Descriptions via Substructure-aware Alignmentcited by 2
  23. 2024DeepProtein: Deep Learning Library and Benchmark for Protein Sequence Learningcited by 1
  24. 2025MolGround: A Benchmark for Molecular Groundingcited by 1
  25. 2025Round-trip Reinforcement Learning: Self-Consistent Training for Better Chemical LLMscited by 1
  26. 2026Entropy-Guided Dynamic Tokens for Graph-LLM Alignment in Molecular Understandingcited by 1
  27. 2024On Machine Learning Approaches for Protein-Ligand Binding Affinity Predictioncited by 0
  28. 2025Automatic Annotation Augmentation Boosts Translation between Molecules and Natural Languagecited by 0
  29. 2025PharMolixFM: All-Atom Foundation Models for Molecular Modeling and Generationcited by 0
  30. 2025Assay2Mol: large language model-based drug design using BioAssay contextcited by 0
  31. 2025HSA-Net: Hierarchical and Structure-Aware Framework for Efficient and Scalable Molecular Language Modelingcited by 0
  32. 2025OsmT: Bridging OpenStreetMap Queries and Natural Language with Open-source Tag-aware Language Modelscited by 0
  33. 2026Chem4DLLM: 4D Multimodal LLMs for Chemical Dynamics Understandingcited by 0
  34. 2026RetroReasoner: A Reasoning LLM for Strategic Retrosynthesis Predictioncited by 0
  35. 2026A Multitask Large Reasoning Model for Molecular Sciencecited by 0
  36. 2026Enhanced Drug-drug Interaction Prediction Using Adaptive Knowledge Integrationcited by 0
  37. 2026The limits of bio-molecular modeling with large language models : a cross-scale evaluationcited by 0
  38. 2026TheBioCollection: Unified Pre-Training Scale LLM Corpus for Biologycited by 0

50 most cited of 54 All 54 (JSON)