# ProtIR: Iterative Refinement between Retrievers and Predictors for Protein Function Annotation

arXiv: https://arxiv.org/abs/2402.07955 · HTML: https://mlsanity.com/paper/2402.07955/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 88 references that are in the map.
- First posted on arXiv 10 Feb 2024.

## Builds on (most cited first, 25 of 25)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Learning from Protein Structure with Geometric Vector Perceptrons](https://mlsanity.com/paper/2009.01411/) (2020)
- [Interpolation Consistency Training for Semi-Supervised Learning](https://mlsanity.com/paper/1903.03825/) (2019)
- [Semi-Supervised Learning with Ladder Networks](https://mlsanity.com/paper/1507.02672/) (2015)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning](https://mlsanity.com/paper/1803.04765/) (2018)
- [ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing](https://mlsanity.com/paper/2007.06225/) (2020)
- [Protein Representation Learning by Geometric Structure Pretraining](https://mlsanity.com/paper/2203.06125/) (2022)
- [Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks](https://mlsanity.com/paper/1511.06390/) (2015)
- [DiffusionRet: Generative Text-Video Retrieval with Diffusion Model](https://mlsanity.com/paper/2303.09867/) (2023)
- [DeepSF: deep convolutional neural network for mapping protein sequences to folds](https://mlsanity.com/paper/1706.01010/) (2017)
- [TorchDrug: A Powerful and Flexible Machine Learning Platform for Drug Discovery](https://mlsanity.com/paper/2202.08320/) (2022)
- [Ankh: Optimized Protein Language Model Unlocks General-Purpose Modelling](https://mlsanity.com/paper/2301.06568/) (2023)
- [Structure-aware Protein Self-supervised Learning](https://mlsanity.com/paper/2204.04213/) (2022)
- [DeepGO: Predicting protein functions from sequence and interactions using a deep ontology-aware classifier](https://mlsanity.com/paper/1705.05919/) (2017)
- [A Systematic Study of Joint Representation Learning on Protein Sequences and Structures](https://mlsanity.com/paper/2303.06275/) (2023)
- [Pre-Training Protein Encoder via Siamese Sequence-Structure Diffusion Trajectory Prediction](https://mlsanity.com/paper/2301.12068/) (2023)
- [Retrieved Sequence Augmentation for Protein Representation Learning](https://mlsanity.com/paper/2302.12563/) (2023)

## Abstract (arXiv)

Protein function annotation is an important yet challenging task in biology. Recent deep learning advancements show significant potential for accurate function prediction by learning from protein sequences and structures. Nevertheless, these predictor-based methods often overlook the modeling of protein similarity, an idea commonly employed in traditional approaches using sequence or structure retrieval tools. To fill this gap, we first study the effect of inter-protein similarity modeling by benchmarking retriever-based methods against predictors on protein function annotation tasks. Our results show that retrievers can match or outperform predictors without large-scale pre-training. Building on these insights, we introduce a novel variational pseudo-likelihood framework, ProtIR, designed to improve function predictors by incorporating inter-protein similarity modeling. This framework iteratively refines knowledge between a function predictor and retriever, thereby combining the strengths of both predictors and retrievers. ProtIR showcases around 10% improvement over vanilla predictor-based methods. Besides, it achieves performance on par with protein language model-based methods, yet without the need for massive pre-training, highlighting the efficacy of our framework. Code will be released upon acceptance.