mlsanityData to 7 Oct 2026

Protein Representation Learning with Sequence Information Embedding: Does it Always Lead to a Better Performance?

arXiv 2406.19755 ↗First posted 28 Jun 2024q-bio.QM cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Deep learning has become a crucial tool in studying proteins. While the significance of modeling protein structure has been discussed extensively in the literature, amino acid types are typically included in the input as a default operation for many inference tasks. This study demonstrates with structure alignment task that embedding amino acid types in some cases may not help a deep learning model learn better representation. To this end, we propose ProtLOCA, a local geometry alignment method based solely on amino acid structure representation. The effectiveness of ProtLOCA is examined by a global structure-matching task on protein pairs with an independent test dataset based on CATH labels. Our method outperforms existing sequence- and structure-based representation learning methods by more quickly and accurately matching structurally consistent protein domains. Furthermore, in local structure pairing tasks, ProtLOCA for the first time provides a valid solution to highlight common local structures among proteins with different overall structures but the same function. This suggests a new possibility for using deep learning methods to analyze protein structure to infer function.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 11 of its 42 references that are in the map. First posted on arXiv 28 Jun 2024.

Builds on

11 of 42 references.

References · 11

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  6. 2021E(n) Equivariant Graph Neural Networkscited by 951
  7. 2020Learning from Protein Structure with Geometric Vector Perceptronscited by 308
  8. 2020ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computingcited by 162
  9. 2022Protein Representation Learning by Geometric Structure Pretrainingcited by 161
  10. 2021Structure-aware Interactive Graph Neural Networks for the Prediction of Protein-Ligand Binding Affinitycited by 60
  11. 2023Ankh: Optimized Protein Language Model Unlocks General-Purpose Modellingcited by 44

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.