# ProLLaMA: A Protein Large Language Model for Multi-Task Protein Language Processing

arXiv: https://arxiv.org/abs/2402.16445 · HTML: https://mlsanity.com/paper/2402.16445/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2024.

## Built on by (most cited first, 30 of 53)

- [Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents](https://mlsanity.com/paper/2503.24047/) (2025)
- [A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery](https://mlsanity.com/paper/2406.10833/) (2024)
- [From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery](https://mlsanity.com/paper/2508.14111/) (2025)
- [A Systematic Survey on Large Language Models for Algorithm Design](https://mlsanity.com/paper/2410.14716/) (2024)
- [Low-Rank Adaptation for Foundation Models: A Comprehensive Review](https://mlsanity.com/paper/2501.00365/) (2024)
- [LLM4Mat-Bench: Benchmarking Large Language Models for Materials Property Prediction](https://mlsanity.com/paper/2411.00177/) (2024)
- [SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Tasks](https://mlsanity.com/paper/2410.03769/) (2024)
- [Protein Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2502.17504/) (2025)
- [Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Survey](https://mlsanity.com/paper/2403.01528/) (2024)
- [Autoregressive + Chain of Thought = Recurrent: Recurrence's Role in Language Models' Computability and a Revisit of Recurrent Transformer](https://mlsanity.com/paper/2409.09239/) (2024)
- [Diffusion on language model encodings for protein sequence generation](https://mlsanity.com/paper/2403.03726/) (2024)
- [A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving](https://mlsanity.com/paper/2509.08269/) (2025)
- [Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models](https://mlsanity.com/paper/2412.19191/) (2024)
- [TourSynbio: A Multi-Modal Large Model and Agent Framework to Bridge Text and Protein Sequences for Protein Engineering](https://mlsanity.com/paper/2408.15299/) (2024)
- [IFAR: Multi-Perspective and Multi-Level Causal Discovery with LLMs](https://mlsanity.com/paper/2409.05559/) (2024)
- [EvoLlama: Enhancing LLMs' Understanding of Proteins via Multimodal Structure and Sequence Representations](https://mlsanity.com/paper/2412.11618/) (2024)
- [Benchmarking Large Language Models on Multiple Tasks in Bioinformatics NLP with Prompting](https://mlsanity.com/paper/2503.04013/) (2025)
- [Steering Protein Language Models](https://mlsanity.com/paper/2509.07983/) (2025)
- [Computational Protein Science in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2501.10282/) (2025)
- [How to Detect and Defeat Molecular Mirage: A Metric-Driven Benchmark for Hallucination in LLM-based Molecular Comprehension](https://mlsanity.com/paper/2504.12314/) (2025)
- [Agentic reinforcement learning empowers next-generation chemical language models for molecular design and synthesis](https://mlsanity.com/paper/2601.17687/) (2026)
- [Proteo-R1: Reasoning Foundation Models for De Novo Protein Design](https://mlsanity.com/paper/2605.02937/) (2026)
- [Controllable Protein Sequence Generation with LLM Preference Optimization](https://mlsanity.com/paper/2501.15007/) (2025)
- [Swarms of Large Language Model Agents for Protein Sequence Design with Experimental Validation](https://mlsanity.com/paper/2511.22311/) (2025)
- [Structure-Enhanced Protein Instruction Tuning: Towards General-Purpose Protein Understanding with LLMs](https://mlsanity.com/paper/2410.03553/) (2024)
- [ProtTeX: Structure-In-Context Reasoning and Editing of Proteins with Large Language Models](https://mlsanity.com/paper/2503.08179/) (2025)
- [Rethinking Text-based Protein Understanding: Retrieval or LLM?](https://mlsanity.com/paper/2505.20354/) (2025)
- [PFMBench: Protein Foundation Model Benchmark](https://mlsanity.com/paper/2506.14796/) (2025)
- [Advances in Protein Representation Learning: Methods, Applications, and Future Directions](https://mlsanity.com/paper/2503.16659/) (2025)
- [MTPNet: Multi-Grained Target Perception for Unified Activity Cliff Prediction](https://mlsanity.com/paper/2506.05427/) (2025)

## Abstract (arXiv)

Recent advances in Protein Language Models (PLMs) have transformed protein engineering, yet unlike their counterparts in Natural Language Processing (NLP), current PLMs exhibit a fundamental limitation: they excel in either Protein Language Understanding (PLU) or Protein Language Generation (PLG), but rarely both. This fragmentation hinders progress in protein engineering. To bridge this gap, we introduce ProLLaMA, a multitask protein language model enhanced by the Evolutionary Protein Generation Framework (EPGF). We construct a comprehensive instruction dataset containing approximately 13 million samples with over 11,000 superfamily annotations to facilitate better modeling of sequence-function landscapes. We leverage a two-stage training approach to develop ProLLaMA, a multitask LLM with protein domain expertise. Our EPGF addresses the mismatch between statistic language modeling and biological constraints through three innovations: a multi-dimensional interpretable scorer, hierarchical efficient decoding, and a probabilistic-biophysical joint selection mechanism. Extensive experiments demonstrate that ProLLaMA excels in both unconditional and controllable protein generation tasks, achieving superior structural quality metrics compared to existing PLMs. Additionally, ProLLaMA demonstrates strong understanding capabilities with a 67.1% exact match rate in superfamily prediction. EPGF significantly enhances the biological viability of generated sequences, as evidenced by improved biophysical scores (+4.3%) and structural metrics (+14.5%). The project is available at https://github.com/PKU-YuanGroup/ProLLaMA.