# InstructProtein: Aligning Human and Protein Language via Knowledge Instruction

arXiv: https://arxiv.org/abs/2310.03269 · HTML: https://mlsanity.com/paper/2310.03269/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 85 references that are in the map.
- First posted on arXiv 5 Oct 2023.

## Builds on (most cited first, 30 of 44)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [A Generalist Agent](https://mlsanity.com/paper/2205.06175/) (2022)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering](https://mlsanity.com/paper/2203.14371/) (2022)
- [Language Is Not All You Need: Aligning Perception with Language Models](https://mlsanity.com/paper/2302.14045/) (2023)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor](https://mlsanity.com/paper/2212.09689/) (2022)
- [DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking](https://mlsanity.com/paper/2210.01776/) (2022)
- [K-BERT: Enabling Language Representation with Knowledge Graph](https://mlsanity.com/paper/1909.07606/) (2019)
- [ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation](https://mlsanity.com/paper/2107.02137/) (2021)
- [MultiModal-GPT: A Vision and Language Model for Dialogue with Humans](https://mlsanity.com/paper/2305.04790/) (2023)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)

## Built on by (most cited first, 28 of 28)

- [Scientific Large Language Models: A Survey on Biological & Chemical Domains](https://mlsanity.com/paper/2401.14656/) (2024)
- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [ProteinGPT: Multimodal LLM for Protein Property Prediction and Structure Understanding](https://mlsanity.com/paper/2408.11363/) (2024)
- [ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training](https://mlsanity.com/paper/2403.07920/) (2024)
- [Protein Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2502.17504/) (2025)
- [Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Survey](https://mlsanity.com/paper/2403.01528/) (2024)
- [Logical Consistency of Large Language Models in Fact-checking](https://mlsanity.com/paper/2412.16100/) (2024)
- [PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomes](https://mlsanity.com/paper/2406.13193/) (2024)
- [Large Knowledge Model: Perspectives and Challenges](https://mlsanity.com/paper/2312.02706/) (2023)
- [Advancing biomolecular understanding and design following human instructions](https://mlsanity.com/paper/2410.07919/) (2024)
- [Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models](https://mlsanity.com/paper/2412.19191/) (2024)
- [Generative AI for Controllable Protein Sequence Design: A Survey](https://mlsanity.com/paper/2402.10516/) (2024)
- [RNA-GPT: Multimodal Generative System for RNA Sequence Understanding](https://mlsanity.com/paper/2411.08900/) (2024)
- [Benchmarking Large Language Models on Multiple Tasks in Bioinformatics NLP with Prompting](https://mlsanity.com/paper/2503.04013/) (2025)
- [CARE: a Benchmark Suite for the Classification and Retrieval of Enzymes](https://mlsanity.com/paper/2406.15669/) (2024)
- [Computational Protein Science in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2501.10282/) (2025)
- [Controllable Protein Sequence Generation with LLM Preference Optimization](https://mlsanity.com/paper/2501.15007/) (2025)
- [Bridging Text and Molecule: A Survey on Multimodal Frameworks for Molecule](https://mlsanity.com/paper/2403.13830/) (2024)
- [Structure-Enhanced Protein Instruction Tuning: Towards General-Purpose Protein Understanding with LLMs](https://mlsanity.com/paper/2410.03553/) (2024)
- [LLM+KG@VLDB'24 Workshop Summary](https://mlsanity.com/paper/2410.01978/) (2024)
- [Prot2Chat: Protein LLM with Early-Fusion of Text, Sequence and Structure](https://mlsanity.com/paper/2502.06846/) (2025)
- [Biological Sequence with Language Model Prompting: A Survey](https://mlsanity.com/paper/2503.04135/) (2025)
- [Prot2Token: A Unified Framework for Protein Modeling via Next-Token Prediction](https://mlsanity.com/paper/2505.20589/) (2025)
- [Enhancing Safe and Controllable Protein Generation via Knowledge Preference Optimization](https://mlsanity.com/paper/2507.10923/) (2025)
- [AFD-INSTRUCTION: A Comprehensive Antibody Instruction Dataset with Functional Annotations for LLM-Based Understanding and Design](https://mlsanity.com/paper/2602.04916/) (2026)
- [Can linguists better understand DNA?](https://mlsanity.com/paper/2412.07678/) (2024)
- [Spatial Knowledge Graph-Guided Multimodal Synthesis](https://mlsanity.com/paper/2505.22633/) (2025)
- [InstructPro2: Natural Language and Biological Grounding Guided De Novo Protein Design](https://mlsanity.com/paper/2610.19592/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have revolutionized the field of natural language processing, but they fall short in comprehending biological sequences such as proteins. To address this challenge, we propose InstructProtein, an innovative LLM that possesses bidirectional generation capabilities in both human and protein languages: (i) taking a protein sequence as input to predict its textual function description and (ii) using natural language to prompt protein sequence generation. To achieve this, we first pre-train an LLM on both protein and natural language corpora, enabling it to comprehend individual languages. Then supervised instruction tuning is employed to facilitate the alignment of these two distinct languages. Herein, we introduce a knowledge graph-based instruction generation framework to construct a high-quality instruction dataset, addressing annotation imbalance and instruction deficits in existing protein-text corpus. In particular, the instructions inherit the structural relations between proteins and function annotations in knowledge graphs, which empowers our model to engage in the causal modeling of protein functions, akin to the chain-of-thought processes in natural languages. Extensive experiments on bidirectional protein-text generation tasks show that InstructProtein outperforms state-of-the-art LLMs by large margins. Moreover, InstructProtein serves as a pioneering step towards text-based protein function prediction and sequence design, effectively bridging the gap between protein and human language understanding.