Protein Representation Learning with Sequence Information Embedding: Does it Always Lead to a Better Performance?
Not on the map. Why?
Abstract (arXiv)
Deep learning has become a crucial tool in studying proteins. While the significance of modeling protein structure has been discussed extensively in the literature, amino acid types are typically included in the input as a default operation for many inference tasks. This study demonstrates with structure alignment task that embedding amino acid types in some cases may not help a deep learning model learn better representation. To this end, we propose ProtLOCA, a local geometry alignment method based solely on amino acid structure representation. The effectiveness of ProtLOCA is examined by a global structure-matching task on protein pairs with an independent test dataset based on CATH labels. Our method outperforms existing sequence- and structure-based representation learning methods by more quickly and accurately matching structurally consistent protein domains. Furthermore, in local structure pairing tasks, ProtLOCA for the first time provides a valid solution to highlight common local structures among proteins with different overall structures but the same function. This suggests a new possibility for using deep learning methods to analyze protein structure to infer function.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 11 of its 42 references that are in the map. First posted on arXiv 28 Jun 2024.
Builds on
11 of 42 references.
References · 11
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
- 2021E(n) Equivariant Graph Neural Networkscited by 951
- 2020Learning from Protein Structure with Geometric Vector Perceptronscited by 308
- 2020ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computingcited by 162
- 2022Protein Representation Learning by Geometric Structure Pretrainingcited by 161
- 2021Structure-aware Interactive Graph Neural Networks for the Prediction of Protein-Ligand Binding Affinitycited by 60
- 2023Ankh: Optimized Protein Language Model Unlocks General-Purpose Modellingcited by 44
Built on by
Citing papers per year: 2024: 1; 2025: 3.
4 papers.
- 2024Retrieval-Enhanced Mutation Mastery: Augmenting Zero-Shot Prediction of Protein Language Modelcited by 7
- 2025VenusX: Unlocking Fine-Grained Functional Understanding of Proteinscited by 5
- 2025Topotein: Topological Deep Learning for Protein Representation Learningcited by 2
- 2025Fast and Interpretable Protein Substructure Alignment via Optimal Transportcited by 1
- 2025Fast and Interpretable Protein Substructure Alignment via Optimal Transport12 Oct 2025
- 2025Topotein: Topological Deep Learning for Protein Representation Learning4 Sep 2025
- 2025VenusX: Unlocking Fine-Grained Functional Understanding of Proteins17 May 2025
- 2024Retrieval-Enhanced Mutation Mastery: Augmenting Zero-Shot Prediction of Protein Language Model28 Oct 2024