# A Survey on Protein Representation Learning: Retrospect and Prospect

arXiv: https://arxiv.org/abs/2301.00813 · HTML: https://mlsanity.com/paper/2301.00813/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 124 references that are in the map.
- First posted on arXiv 31 Dec 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Learning from Protein Structure with Geometric Vector Perceptrons](https://mlsanity.com/paper/2009.01411/) (2020)
- [Graph Self-Supervised Learning: A Survey](https://mlsanity.com/paper/2103.00111/) (2021)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [Structure-based Drug Design with Equivariant Diffusion Models](https://mlsanity.com/paper/2210.13695/) (2022)
- [Self-Supervised Learning of Graph Neural Networks: A Unified Review](https://mlsanity.com/paper/2102.10757/) (2021)
- [Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pockets](https://mlsanity.com/paper/2205.07249/) (2022)
- [ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing](https://mlsanity.com/paper/2007.06225/) (2020)
- [Protein Representation Learning by Geometric Structure Pretraining](https://mlsanity.com/paper/2203.06125/) (2022)
- [A 3D Generative Model for Structure-Based Drug Design](https://mlsanity.com/paper/2203.10446/) (2022)
- [Self-supervised Learning on Graphs: Contrastive, Generative,or Predictive](https://mlsanity.com/paper/2105.07342/) (2021)
- [ProGen: Language Modeling for Protein Generation](https://mlsanity.com/paper/2004.03497/) (2020)
- [Generating 3D Molecules for Target Protein Binding](https://mlsanity.com/paper/2204.09410/) (2022)
- [DiffBP: Generative Diffusion of 3D Molecules for Target Protein Binding](https://mlsanity.com/paper/2211.11214/) (2022)
- [Learning protein sequence embeddings using information from structure](https://mlsanity.com/paper/1902.08661/) (2019)
- [End-to-End Learning on 3D Protein Structure for Interface Prediction](https://mlsanity.com/paper/1807.01297/) (2018)
- [Generating 3D Molecular Structures Conditional on a Receptor Binding Site with Deep Generative Models](https://mlsanity.com/paper/2010.14442/) (2020)
- [Structure-aware Protein Self-supervised Learning](https://mlsanity.com/paper/2204.04213/) (2022)
- [Is Transfer Learning Necessary for Protein Landscape Prediction?](https://mlsanity.com/paper/2011.03443/) (2020)
- [Modeling Protein Using Large-scale Pretrain Language Model](https://mlsanity.com/paper/2108.07435/) (2021)
- [Pre-training Co-evolutionary Protein Representation via A Pairwise Masked Language Model](https://mlsanity.com/paper/2110.15527/) (2021)
- [Pre-Training of Deep Bidirectional Protein Sequence Representations with Structural Information](https://mlsanity.com/paper/1912.05625/) (2019)
- [Profile Prediction: An Alignment-Based Pre-Training Task for Protein Sequence Models](https://mlsanity.com/paper/2012.00195/) (2020)
- [Variational auto-encoding of protein sequences](https://mlsanity.com/paper/1712.03346/) (2017)

## Built on by (most cited first, 10 of 10)

- [A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applications](https://mlsanity.com/paper/2403.00485/) (2024)
- [PPFlow: Target-aware Peptide Design with Torsional Flow Matching](https://mlsanity.com/paper/2405.06642/) (2024)
- [MAPE-PPI: Towards Effective and Efficient Protein-Protein Interaction Prediction via Microenvironment-Aware Protein Embedding](https://mlsanity.com/paper/2402.14391/) (2024)
- [Data-Efficient Protein 3D Geometric Pretraining via Refinement of Diffused Protein Structure Decoy](https://mlsanity.com/paper/2302.10888/) (2023)
- [Computational Protein Science in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2501.10282/) (2025)
- [Interpretable machine learning of amino acid patterns in proteins: a statistical ensemble approach](https://mlsanity.com/paper/2303.15228/) (2023)
- [Advances in Protein Representation Learning: Methods, Applications, and Future Directions](https://mlsanity.com/paper/2503.16659/) (2025)
- [Enhancing Protein Predictive Models via Proteins Data Augmentation: A Benchmark and New Directions](https://mlsanity.com/paper/2403.00875/) (2024)
- [BioBlobs: Unsupervised Discovery of Functional Substructures for Protein Function Prediction](https://mlsanity.com/paper/2510.01632/) (2025)
- [RIGA-Fold: A General Framework for Protein Inverse Folding via Recurrent Interaction and Geometric Awareness](https://mlsanity.com/paper/2602.04637/) (2026)

## Abstract (arXiv)

Proteins are fundamental biological entities that play a key role in life activities. The amino acid sequences of proteins can be folded into stable 3D structures in the real physicochemical world, forming a special kind of sequence-structure data. With the development of Artificial Intelligence (AI) techniques, Protein Representation Learning (PRL) has recently emerged as a promising research topic for extracting informative knowledge from massive protein sequences or structures. To pave the way for AI researchers with little bioinformatics background, we present a timely and comprehensive review of PRL formulations and existing PRL methods from the perspective of model architectures, pretext tasks, and downstream applications. We first briefly introduce the motivations for protein representation learning and formulate it in a general and unified framework. Next, we divide existing PRL methods into three main categories: sequence-based, structure-based, and sequence-structure co-modeling. Finally, we discuss some technical challenges and potential directions for improving protein representation learning. The latest advances in PRL methods are summarized in a GitHub repository https://github.com/LirongWu/awesome-protein-representation-learning.