# PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understanding

arXiv: https://arxiv.org/abs/2206.02096 · HTML: https://mlsanity.com/paper/2206.02096/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 132 references that are in the map.
- First posted on arXiv 5 Jun 2022.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization](https://mlsanity.com/paper/1908.01000/) (2019)
- [Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development](https://mlsanity.com/paper/2102.09548/) (2021)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing](https://mlsanity.com/paper/2007.06225/) (2020)
- [Protein Representation Learning by Geometric Structure Pretraining](https://mlsanity.com/paper/2203.06125/) (2022)
- [ATOM3D: Tasks On Molecules in Three Dimensions](https://mlsanity.com/paper/2012.04035/) (2020)
- [DeepAffinity: Interpretable Deep Learning of Compound-Protein Affinity through Unified Recurrent and Convolutional Neural Networks](https://mlsanity.com/paper/1806.07537/) (2018)
- [DeepSF: deep convolutional neural network for mapping protein sequences to folds](https://mlsanity.com/paper/1706.01010/) (2017)
- [TorchDrug: A Powerful and Flexible Machine Learning Platform for Drug Discovery](https://mlsanity.com/paper/2202.08320/) (2022)
- [ProteinNet: a standardized data set for machine learning of protein structure](https://mlsanity.com/paper/1902.00249/) (2019)
- [Contrastive Representation Learning for 3D Protein Structures](https://mlsanity.com/paper/2205.15675/) (2022)
- [Is Transfer Learning Necessary for Protein Landscape Prediction?](https://mlsanity.com/paper/2011.03443/) (2020)
- [Pre-training Co-evolutionary Protein Representation via A Pairwise Masked Language Model](https://mlsanity.com/paper/2110.15527/) (2021)
- [Pre-training of Graph Neural Network for Modeling Effects of Mutations on Protein-Protein Binding Affinity](https://mlsanity.com/paper/2008.12473/) (2020)
- [SS-GNN: A Simple-Structured Graph Neural Network for Affinity Prediction](https://mlsanity.com/paper/2206.07015/) (2022)
- [Distributed Representations for Biological Sequence Analysis](https://mlsanity.com/paper/1608.05949/) (2016)

## Built on by (most cited first, 30 of 77)

- [BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations](https://mlsanity.com/paper/2310.07276/) (2023)
- [Scientific Large Language Models: A Survey on Biological & Chemical Domains](https://mlsanity.com/paper/2401.14656/) (2024)
- [ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts](https://mlsanity.com/paper/2301.12040/) (2023)
- [BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuning](https://mlsanity.com/paper/2402.17810/) (2024)
- [Ankh: Optimized Protein Language Model Unlocks General-Purpose Modelling](https://mlsanity.com/paper/2301.06568/) (2023)
- [BEND: Benchmarking DNA Language Models on biologically meaningful tasks](https://mlsanity.com/paper/2311.12570/) (2023)
- [A Survey of Generative AI for de novo Drug Design: New Frontiers in Molecule and Protein Generation](https://mlsanity.com/paper/2402.08703/) (2024)
- [A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applications](https://mlsanity.com/paper/2403.00485/) (2024)
- [ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training](https://mlsanity.com/paper/2403.07920/) (2024)
- [A Fine-tuning Dataset and Benchmark for Large Language Models for Protein Understanding](https://mlsanity.com/paper/2406.05540/) (2024)
- [SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Models](https://mlsanity.com/paper/2406.09098/) (2024)
- [Protein Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2502.17504/) (2025)
- [Protein Language Models and Structure Prediction: Connection and Progression](https://mlsanity.com/paper/2211.16742/) (2022)
- [Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Survey](https://mlsanity.com/paper/2403.01528/) (2024)
- [Evaluating representation learning on the protein structure universe](https://mlsanity.com/paper/2406.13864/) (2024)
- [A Systematic Study of Joint Representation Learning on Protein Sequences and Structures](https://mlsanity.com/paper/2303.06275/) (2023)
- [Protein Structure Tokenization: Benchmarking and New Recipe](https://mlsanity.com/paper/2503.00089/) (2025)
- [Offline Model-Based Optimization: Comprehensive Review](https://mlsanity.com/paper/2503.17286/) (2025)
- [MatSciML: A Broad, Multi-Task Benchmark for Solid-State Materials Modeling](https://mlsanity.com/paper/2309.05934/) (2023)
- [A Survey of Graph Transformers: Architectures, Theories and Applications](https://mlsanity.com/paper/2502.16533/) (2025)
- [Pre-Training Protein Encoder via Siamese Sequence-Structure Diffusion Trajectory Prediction](https://mlsanity.com/paper/2301.12068/) (2023)
- [Kermut: Composite kernel regression for protein variant effects](https://mlsanity.com/paper/2407.00002/) (2024)
- [Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Tasks](https://mlsanity.com/paper/2410.22296/) (2024)
- [Annotation-guided Protein Design with Multi-Level Domain Alignment](https://mlsanity.com/paper/2404.16866/) (2024)
- [Closed-Form Test Functions for Biophysical Sequence Optimization Algorithms](https://mlsanity.com/paper/2407.00236/) (2024)
- [Benchmarking Large Language Models on Multiple Tasks in Bioinformatics NLP with Prompting](https://mlsanity.com/paper/2503.04013/) (2025)
- [Linguistically inspired roadmap for building biologically reliable protein language models](https://mlsanity.com/paper/2207.00982/) (2022)
- [Retrieved Sequence Augmentation for Protein Representation Learning](https://mlsanity.com/paper/2302.12563/) (2023)
- [BEACON: Benchmark for Comprehensive RNA Tasks and Language Models](https://mlsanity.com/paper/2406.10391/) (2024)
- [Structure-Informed Protein Language Model](https://mlsanity.com/paper/2402.05856/) (2024)

## Abstract (arXiv)

We are now witnessing significant progress of deep learning methods in a variety of tasks (or datasets) of proteins. However, there is a lack of a standard benchmark to evaluate the performance of different methods, which hinders the progress of deep learning in this field. In this paper, we propose such a benchmark called PEER, a comprehensive and multi-task benchmark for Protein sEquence undERstanding. PEER provides a set of diverse protein understanding tasks including protein function prediction, protein localization prediction, protein structure prediction, protein-protein interaction prediction, and protein-ligand interaction prediction. We evaluate different types of sequence-based methods for each task including traditional feature engineering approaches, different sequence encoding methods as well as large-scale pre-trained protein language models. In addition, we also investigate the performance of these methods under the multi-task learning setting. Experimental results show that large-scale pre-trained protein language models achieve the best performance for most individual tasks, and jointly training multiple tasks further boosts the performance. The datasets and source codes of this benchmark are all available at https://github.com/DeepGraphLearning/PEER_Benchmark