# SESNet: sequence-structure feature-integrated deep learning method for data-efficient protein engineering

arXiv: https://arxiv.org/abs/2301.00004 · HTML: https://mlsanity.com/paper/2301.00004/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 47 references that are in the map.
- First posted on arXiv 29 Dec 2022.

## Builds on (most cited first, 6 of 6)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing](https://mlsanity.com/paper/2007.06225/) (2020)
- [Protein Representation Learning by Geometric Structure Pretraining](https://mlsanity.com/paper/2203.06125/) (2022)
- [ProGen2: Exploring the Boundaries of Protein Language Models](https://mlsanity.com/paper/2206.13517/) (2022)

## Built on by (most cited first, 5 of 5)

- [A Text-guided Protein Design Framework](https://mlsanity.com/paper/2302.04611/) (2023)
- [Simple, Efficient and Scalable Structure-aware Adapter Boosts Protein Language Models](https://mlsanity.com/paper/2404.14850/) (2024)
- [Immunogenicity Prediction with Dual Attention Enables Vaccine Target Selection](https://mlsanity.com/paper/2410.02647/) (2024)
- [PETA: Evaluating the Impact of Protein Transfer Learning with Sub-word Tokenization on Downstream Applications](https://mlsanity.com/paper/2310.17415/) (2023)
- [CPE-Pro: A Structure-Sensitive Deep Learning Method for Protein Representation and Origin Evaluation](https://mlsanity.com/paper/2410.15592/) (2024)

## Abstract (arXiv)

Deep learning has been widely used for protein engineering. However, it is limited by the lack of sufficient experimental data to train an accurate model for predicting the functional fitness of high-order mutants. Here, we develop SESNet, a supervised deep-learning model to predict the fitness for protein mutants by leveraging both sequence and structure information, and exploiting attention mechanism. Our model integrates local evolutionary context from homologous sequences, the global evolutionary context encoding rich semantic from the universal protein sequence space and the structure information accounting for the microenvironment around each residue in a protein. We show that SESNet outperforms state-of-the-art models for predicting the sequence-function relationship on 26 deep mutational scanning datasets. More importantly, we propose a data augmentation strategy by leveraging the data from unsupervised models to pre-train our model. After that, our model can achieve strikingly high accuracy in prediction of the fitness of protein mutants, especially for the higher order variants (> 4 mutation sites), when finetuned by using only a small number of experimental mutation data (<50). The strategy proposed is of great practical value as the required experimental effort, i.e., producing a few tens of experimental mutation data on a given protein, is generally affordable by an ordinary biochemical group and can be applied on almost any protein.