# Is Transfer Learning Necessary for Protein Landscape Prediction?

arXiv: https://arxiv.org/abs/2011.03443 · HTML: https://mlsanity.com/paper/2011.03443/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2020.

## Built on by (most cited first, 28 of 28)

- [Protein Representation Learning by Geometric Structure Pretraining](https://mlsanity.com/paper/2203.06125/) (2022)
- [ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Texts](https://mlsanity.com/paper/2301.12040/) (2023)
- [PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understanding](https://mlsanity.com/paper/2206.02096/) (2022)
- [Protein sequence design with deep generative models](https://mlsanity.com/paper/2104.04457/) (2021)
- [Contrastive Representation Learning for 3D Protein Structures](https://mlsanity.com/paper/2205.15675/) (2022)
- [Generalist Equivariant Transformer Towards 3D Molecular Interaction Learning](https://mlsanity.com/paper/2306.01474/) (2023)
- [A Systematic Study of Joint Representation Learning on Protein Sequences and Structures](https://mlsanity.com/paper/2303.06275/) (2023)
- [Endowing Protein Language Models with Structural Knowledge](https://mlsanity.com/paper/2401.14819/) (2024)
- [SciReasoner: Laying the Scientific Reasoning Ground Across Disciplines](https://mlsanity.com/paper/2509.21320/) (2025)
- [A Survey on Protein Representation Learning: Retrospect and Prospect](https://mlsanity.com/paper/2301.00813/) (2022)
- [Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models](https://mlsanity.com/paper/2412.19191/) (2024)
- [Annotation-guided Protein Design with Multi-Level Domain Alignment](https://mlsanity.com/paper/2404.16866/) (2024)
- [EvoLlama: Enhancing LLMs' Understanding of Proteins via Multimodal Structure and Sequence Representations](https://mlsanity.com/paper/2412.11618/) (2024)
- [BEACON: Benchmark for Comprehensive RNA Tasks and Language Models](https://mlsanity.com/paper/2406.10391/) (2024)
- [Simple, Efficient and Scalable Structure-aware Adapter Boosts Protein Language Models](https://mlsanity.com/paper/2404.14850/) (2024)
- [ProtCLIP: Function-Informed Protein Multi-Modal Learning](https://mlsanity.com/paper/2412.20014/) (2024)
- [GlycanML: A Multi-Task and Multi-Structure Benchmark for Glycan Machine Learning](https://mlsanity.com/paper/2405.16206/) (2024)
- [SeqProFT: Sequence-only Protein Property Prediction with LoRA Finetuning](https://mlsanity.com/paper/2411.11530/) (2024)
- [Deep Manifold Transformation for Protein Representation Learning](https://mlsanity.com/paper/2402.09416/) (2024)
- [DeepProtein: Deep Learning Library and Benchmark for Protein Sequence Learning](https://mlsanity.com/paper/2410.02023/) (2024)
- [Learning Representations for Reasoning: Generalizing Across Diverse Structures](https://mlsanity.com/paper/2410.13018/) (2024)
- [CAPSUL: A Comprehensive Human Protein Benchmark for Subcellular Localization](https://mlsanity.com/paper/2603.18571/) (2026)
- [BioMatrix: Towards a Comprehensive Biological Foundation Model Spanning the Modality Matrix of Sequences, Structures, and Language](https://mlsanity.com/paper/2606.22138/) (2026)
- [Random Embeddings and Linear Regression can Predict Protein Function](https://mlsanity.com/paper/2104.14661/) (2021)
- [CPE-Pro: A Structure-Sensitive Deep Learning Method for Protein Representation and Origin Evaluation](https://mlsanity.com/paper/2410.15592/) (2024)
- [Modeling All-Atom Glycan Structures via Hierarchical Message Passing and Multi-Scale Pre-training](https://mlsanity.com/paper/2506.01376/) (2025)
- [A Novel Framework for Multi-Modal Protein Representation Learning](https://mlsanity.com/paper/2510.23273/) (2025)
- [Enhancing Protein Representation Learning via Manifold Restore Mixing](https://mlsanity.com/paper/2606.22307/) (2026)

## Abstract (arXiv)

Recently, there has been great interest in learning how to best represent proteins, specifically with fixed-length embeddings. Deep learning has become a popular tool for protein representation learning as a model's hidden layers produce potentially useful vector embeddings. TAPE introduced a number of benchmark tasks and showed that semi-supervised learning, via pretraining language models on a large protein corpus, improved performance on downstream tasks. Two of the tasks (fluorescence prediction and stability prediction) involve learning fitness landscapes. In this paper, we show that CNN models trained solely using supervised learning both compete with and sometimes outperform the best models from TAPE that leverage expensive pretraining on large protein datasets. These CNN models are sufficiently simple and small that they can be trained using a Google Colab notebook. We also find for the fluorescence task that linear regression outperforms our models and the TAPE models. The benchmarking tasks proposed by TAPE are excellent measures of a model's ability to predict protein function and should be used going forward. However, we believe it is important to add baselines from simple models to put the performance of the semi-supervised models that have been reported so far into perspective.