# Retrieval-Enhanced Mutation Mastery: Augmenting Zero-Shot Prediction of Protein Language Model

arXiv: https://arxiv.org/abs/2410.21127 · HTML: https://mlsanity.com/paper/2410.21127/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 37 references that are in the map.
- First posted on arXiv 28 Oct 2024.

## Builds on (most cited first, 4 of 4)

- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Learning from Protein Structure with Geometric Vector Perceptrons](https://mlsanity.com/paper/2009.01411/) (2020)
- [Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval](https://mlsanity.com/paper/2205.13760/) (2022)
- [Protein Representation Learning with Sequence Information Embedding: Does it Always Lead to a Better Performance?](https://mlsanity.com/paper/2406.19755/) (2024)

## Built on by (most cited first, 7 of 7)

- [Understanding protein function with a multimodal retrieval-augmented foundation model](https://mlsanity.com/paper/2508.04724/) (2025)
- [VenusX: Unlocking Fine-Grained Functional Understanding of Proteins](https://mlsanity.com/paper/2505.11812/) (2025)
- [Rethinking Text-based Protein Understanding: Retrieval or LLM?](https://mlsanity.com/paper/2505.20354/) (2025)
- [Advances in Protein Representation Learning: Methods, Applications, and Future Directions](https://mlsanity.com/paper/2503.16659/) (2025)
- [Sequence-Only Prediction of Binding Affinity Changes: A Robust and Interpretable Model for Antibody Engineering](https://mlsanity.com/paper/2505.20301/) (2025)
- [InstructPLM-mu: 1-Hour Fine-Tuning of ESM2 Beats ESM3 in Protein Mutation Predictions](https://mlsanity.com/paper/2510.03370/) (2025)
- [Evolutionary Profiles for Protein Fitness Prediction](https://mlsanity.com/paper/2510.07286/) (2025)

## Abstract (arXiv)

Enzyme engineering enables the modification of wild-type proteins to meet industrial and research demands by enhancing catalytic activity, stability, binding affinities, and other properties. The emergence of deep learning methods for protein modeling has demonstrated superior results at lower costs compared to traditional approaches such as directed evolution and rational design. In mutation effect prediction, the key to pre-training deep learning models lies in accurately interpreting the complex relationships among protein sequence, structure, and function. This study introduces a retrieval-enhanced protein language model for comprehensive analysis of native properties from sequence and local structural interactions, as well as evolutionary properties from retrieved homologous sequences. The state-of-the-art performance of the proposed ProtREM is validated on over 2 million mutants across 217 assays from an open benchmark (ProteinGym). We also conducted post-hoc analyses of the model's ability to improve the stability and binding affinity of a VHH antibody. Additionally, we designed 10 new mutants on a DNA polymerase and conducted wet-lab experiments to evaluate their enhanced activity at higher temperatures. Both in silico and experimental evaluations confirmed that our method provides reliable predictions of mutation effects, offering an auxiliary tool for biologists aiming to evolve existing enzymes. The implementation is publicly available at https://github.com/tyang816/ProtREM.