# Tx-LLM: A Large Language Model for Therapeutics

arXiv: https://arxiv.org/abs/2406.06316 · HTML: https://mlsanity.com/paper/2406.06316/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 101 references that are in the map.
- First posted on arXiv 10 Jun 2024.

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [ChemCrow: Augmenting large-language models with chemistry tools](https://mlsanity.com/paper/2304.05376/) (2023)
- [Towards Expert-Level Medical Question Answering with Large Language Models](https://mlsanity.com/paper/2305.09617/) (2023)
- [Analyzing Learned Molecular Representations for Property Prediction](https://mlsanity.com/paper/1904.01561/) (2019)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [Leveraging Large Language Models for Multiple Choice Question Answering](https://mlsanity.com/paper/2210.12353/) (2022)
- [What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks](https://mlsanity.com/paper/2305.18365/) (2023)
- [EquiBind: Geometric Deep Learning for Drug Binding Structure Prediction](https://mlsanity.com/paper/2202.05146/) (2022)
- [LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset](https://mlsanity.com/paper/2402.09391/) (2024)
- [BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations](https://mlsanity.com/paper/2310.07276/) (2023)
- [Pathways: Asynchronous Distributed Dataflow for ML](https://mlsanity.com/paper/2203.12533/) (2022)
- [HINT: Hierarchical Interaction Network for Trial Outcome Prediction Leveraging Web Data](https://mlsanity.com/paper/2102.04252/) (2021)
- [Learning Graph-Level Representation for Drug Discovery](https://mlsanity.com/paper/1709.03741/) (2017)
- [ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training](https://mlsanity.com/paper/2403.07920/) (2024)
- [TITAN: T Cell Receptor Specificity Prediction with Bimodal Attention Networks](https://mlsanity.com/paper/2105.03323/) (2021)
- [Combining docking pose rank and structure with deep learning improves protein-ligand binding mode prediction](https://mlsanity.com/paper/1910.02845/) (2019)
- [Otter-Knowledge: benchmarks of multimodal knowledge graph representation learning from different sources for drug discovery](https://mlsanity.com/paper/2306.12802/) (2023)
- [SSM-DTA: Breaking the Barriers of Data Scarcity in Drug-Target Affinity Prediction](https://mlsanity.com/paper/2206.09818/) (2022)
- [Structure to Property: Chemical Element Embeddings and a Deep Learning Approach for Accurate Prediction of Chemical Properties](https://mlsanity.com/paper/2309.09355/) (2023)

## Built on by (most cited first, 25 of 25)

- [A Review of Large Language Models and Autonomous Agents in Chemistry](https://mlsanity.com/paper/2407.01603/) (2024)
- [Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents](https://mlsanity.com/paper/2503.24047/) (2025)
- [Robin: A multi-agent system for automating scientific discovery](https://mlsanity.com/paper/2505.13400/) (2025)
- [LLMs for Explainable AI: A Comprehensive Survey](https://mlsanity.com/paper/2504.00125/) (2025)
- [TxGemma: Efficient and Agentic LLMs for Therapeutics](https://mlsanity.com/paper/2504.06196/) (2025)
- [DrugPilot: LLM-based Parameterized Reasoning Agent for Drug Discovery](https://mlsanity.com/paper/2505.13940/) (2025)
- [RAG-Enhanced Collaborative LLM Agents for Drug Discovery](https://mlsanity.com/paper/2502.17506/) (2025)
- [Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Model](https://mlsanity.com/paper/2502.13449/) (2025)
- [K-Paths: Reasoning over Graph Paths for Drug Repurposing and Drug Interaction Prediction](https://mlsanity.com/paper/2502.13344/) (2025)
- [MAMMAL -- Molecular Aligned Multi-Modal Architecture and Language](https://mlsanity.com/paper/2410.22367/) (2024)
- [Y-Mol: A Multiscale Biomedical Knowledge-Guided Large Language Model for Drug Development](https://mlsanity.com/paper/2410.11550/) (2024)
- [Large Language Models for Bioinformatics](https://mlsanity.com/paper/2501.06271/) (2025)
- [A Japanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical NLP](https://mlsanity.com/paper/2505.16661/) (2025)
- [Case-Based Reasoning Enhances the Predictive Power of LLMs in Drug-Drug Interaction](https://mlsanity.com/paper/2505.23034/) (2025)
- [Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?](https://mlsanity.com/paper/2506.10912/) (2025)
- [PyTDC: A multimodal machine learning training, evaluation, and inference platform for biomedical foundation models](https://mlsanity.com/paper/2505.05577/) (2025)
- [Patient-specific Biomolecular Instruction Tuning](https://mlsanity.com/paper/2509.22853/) (2025)
- [RAGPPI: RAG Benchmark for Protein-Protein Interactions in Drug Discovery](https://mlsanity.com/paper/2505.23823/) (2025)
- [MIMIC: A Generative Multimodal Foundation Model for Biomolecules](https://mlsanity.com/paper/2604.24506/) (2026)
- [Bolek: A Multimodal Language Model for Molecular Reasoning](https://mlsanity.com/paper/2605.02745/) (2026)
- [TheBioCollection: Unified Pre-Training Scale LLM Corpus for Biology](https://mlsanity.com/paper/2607.08803/) (2026)
- [Rigorous Evaluation of Large Language Models for Malaria Drug Discovery: Trade-offs in Performance, Scale, and Resource Utility](https://mlsanity.com/paper/2608.20418/) (2026)
- [DrugReason: Dynamic Multi-View Reasoning over Knowledge Graph and Language Evidence for Drug Repurposing](https://mlsanity.com/paper/2609.06779/) (2026)
- [LLM sequential decision making under uncertainty in biochemical domains](https://mlsanity.com/paper/2609.33061/) (2026)
- [Large Language Models are Approximate Survival Estimators](https://mlsanity.com/paper/2609.38181/) (2026)

## Abstract (arXiv)

Developing therapeutics is a lengthy and expensive process that requires the satisfaction of many different criteria, and AI models capable of expediting the process would be invaluable. However, the majority of current AI approaches address only a narrowly defined set of tasks, often circumscribed within a particular domain. To bridge this gap, we introduce Tx-LLM, a generalist large language model (LLM) fine-tuned from PaLM-2 which encodes knowledge about diverse therapeutic modalities. Tx-LLM is trained using a collection of 709 datasets that target 66 tasks spanning various stages of the drug discovery pipeline. Using a single set of weights, Tx-LLM simultaneously processes a wide variety of chemical or biological entities(small molecules, proteins, nucleic acids, cell lines, diseases) interleaved with free-text, allowing it to predict a broad range of associated properties, achieving competitive with state-of-the-art (SOTA) performance on 43 out of 66 tasks and exceeding SOTA on 22. Among these, Tx-LLM is particularly powerful and exceeds best-in-class performance on average for tasks combining molecular SMILES representations with text such as cell line names or disease names, likely due to context learned during pretraining. We observe evidence of positive transfer between tasks with diverse drug types (e.g.,tasks involving small molecules and tasks involving proteins), and we study the impact of model size, domain finetuning, and prompting strategies on performance. We believe Tx-LLM represents an important step towards LLMs encoding biochemical knowledge and could have a future role as an end-to-end tool across the drug discovery development pipeline.