# BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuning

arXiv: https://arxiv.org/abs/2402.17810 · HTML: https://mlsanity.com/paper/2402.17810/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 89 references that are in the map.
- First posted on arXiv 27 Feb 2024.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [An Introduction to Convolutional Neural Networks](https://mlsanity.com/paper/1511.08458/) (2015)
- [ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction](https://mlsanity.com/paper/2010.09885/) (2020)
- [Pre-training Molecular Graph Representation with 3D Geometry](https://mlsanity.com/paper/2110.07728/) (2021)
- [ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction](https://mlsanity.com/paper/2106.06130/) (2021)
- [Fr'echet ChemNet Distance: A metric for generative models for molecules in drug discovery](https://mlsanity.com/paper/1803.09518/) (2018)
- [Translation between Molecules and Natural Language](https://mlsanity.com/paper/2204.11817/) (2022)
- [Motif-based Graph Self-Supervised Learning for Molecular Property Prediction](https://mlsanity.com/paper/2110.00987/) (2021)
- [Investigating the Limitations of Transformers with Simple Arithmetic Tasks](https://mlsanity.com/paper/2102.13019/) (2021)
- [A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language](https://mlsanity.com/paper/2209.05481/) (2022)
- [MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter](https://mlsanity.com/paper/2310.12798/) (2023)
- [The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4](https://mlsanity.com/paper/2311.07361/) (2023)
- [BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicine](https://mlsanity.com/paper/2308.09442/) (2023)
- [BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations](https://mlsanity.com/paper/2310.07276/) (2023)
- [Unifying Molecular and Textual Representations via Multi-task Language Modelling](https://mlsanity.com/paper/2301.12586/) (2023)
- [Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective](https://mlsanity.com/paper/2306.06615/) (2023)
- [InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery](https://mlsanity.com/paper/2311.16208/) (2023)
- [Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks](https://mlsanity.com/paper/2305.14201/) (2023)
- [GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text](https://mlsanity.com/paper/2308.06911/) (2023)
- [MolXPT: Wrapping Molecules with Text for Generative Pre-training](https://mlsanity.com/paper/2305.10688/) (2023)

## Built on by (most cited first, 30 of 54)

- [A Review of Large Language Models and Autonomous Agents in Chemistry](https://mlsanity.com/paper/2407.01603/) (2024)
- [Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2502.10708/) (2025)
- [SciToolAgent: A Knowledge Graph-Driven Scientific Agent for Multi-Tool Integration](https://mlsanity.com/paper/2507.20280/) (2025)
- [Protein Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2502.17504/) (2025)
- [Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Survey](https://mlsanity.com/paper/2403.01528/) (2024)
- [ChemMLLM: Chemical Multimodal Large Language Model](https://mlsanity.com/paper/2505.16326/) (2025)
- [PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomes](https://mlsanity.com/paper/2406.13193/) (2024)
- [Causal Representation Learning from Multimodal Biomedical Observations](https://mlsanity.com/paper/2411.06518/) (2024)
- [MolReasoner: Toward Effective and Interpretable Reasoning for Molecular LLMs](https://mlsanity.com/paper/2508.02066/) (2025)
- [LDMol: A Text-to-Molecule Diffusion Model with Structurally Informative Latent Space Surpasses AR Models](https://mlsanity.com/paper/2405.17829/) (2024)
- [Learning Multi-view Molecular Representations with Structured and Unstructured Knowledge](https://mlsanity.com/paper/2406.09841/) (2024)
- [Regression with Large Language Models for Materials and Molecular Property Prediction](https://mlsanity.com/paper/2409.06080/) (2024)
- [GP-GPT: Large Language Model for Gene-Phenotype Mapping](https://mlsanity.com/paper/2409.09825/) (2024)
- [SciReasoner: Laying the Scientific Reasoning Ground Across Disciplines](https://mlsanity.com/paper/2509.21320/) (2025)
- [Advancing biomolecular understanding and design following human instructions](https://mlsanity.com/paper/2410.07919/) (2024)
- [3D-MolT5: Leveraging Discrete Structural Information for Molecule-Text Modeling](https://mlsanity.com/paper/2406.05797/) (2024)
- [Can LLMs Generate Diverse Molecules? Towards Alignment with Structural Diversity](https://mlsanity.com/paper/2410.03138/) (2024)
- [EvoLlama: Enhancing LLMs' Understanding of Proteins via Multimodal Structure and Sequence Representations](https://mlsanity.com/paper/2412.11618/) (2024)
- [Enhancing Chemical Reaction and Retrosynthesis Prediction with Large Language Model and Dual-task Learning](https://mlsanity.com/paper/2505.02639/) (2025)
- [Reasoning-Enhanced Large Language Models for Molecular Property Prediction](https://mlsanity.com/paper/2510.10248/) (2025)
- [OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discovery](https://mlsanity.com/paper/2501.15415/) (2025)
- [mCLM: A Modular Chemical Language Model that Generates Functional and Makeable Molecules](https://mlsanity.com/paper/2505.12565/) (2025)
- [Steering Protein Language Models](https://mlsanity.com/paper/2509.07983/) (2025)
- [Computational Protein Science in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2501.10282/) (2025)
- [Mol-LLM: Multimodal Generalist Molecular LLM with Improved Graph Utilization](https://mlsanity.com/paper/2502.02810/) (2025)
- [MolX: Enhancing Large Language Models for Molecular Understanding With A Multi-Modal Extension](https://mlsanity.com/paper/2406.06777/) (2024)
- [Structure-Enhanced Protein Instruction Tuning: Towards General-Purpose Protein Understanding with LLMs](https://mlsanity.com/paper/2410.03553/) (2024)
- [MAMMAL -- Molecular Aligned Multi-Modal Architecture and Language](https://mlsanity.com/paper/2410.22367/) (2024)
- [Rethinking Text-based Protein Understanding: Retrieval or LLM?](https://mlsanity.com/paper/2505.20354/) (2025)
- [Instruction-Based Molecular Graph Generation with Unified Text-Graph Diffusion Model](https://mlsanity.com/paper/2408.09896/) (2024)

## Abstract (arXiv)

Recent research trends in computational biology have increasingly focused on integrating text and bio-entity modeling, especially in the context of molecules and proteins. However, previous efforts like BioT5 faced challenges in generalizing across diverse tasks and lacked a nuanced understanding of molecular structures, particularly in their textual representations (e.g., IUPAC). This paper introduces BioT5+, an extension of the BioT5 framework, tailored to enhance biological research and drug discovery. BioT5+ incorporates several novel features: integration of IUPAC names for molecular understanding, inclusion of extensive bio-text and molecule data from sources like bioRxiv and PubChem, the multi-task instruction tuning for generality across tasks, and a numerical tokenization technique for improved processing of numerical data. These enhancements allow BioT5+ to bridge the gap between molecular representations and their textual descriptions, providing a more holistic understanding of biological entities, and largely improving the grounded reasoning of bio-text and bio-sequences. The model is pre-trained and fine-tuned with a large number of experiments, including 3 types of problems (classification, regression, generation), 15 kinds of tasks, and 21 total benchmark datasets, demonstrating the remarkable performance and state-of-the-art results in most cases. BioT5+ stands out for its ability to capture intricate relationships in biological data, thereby contributing significantly to bioinformatics and computational biology. Our code is available at urlhttps://github.com/QizhiPei/BioT5.