# Random Embeddings and Linear Regression can Predict Protein Function

arXiv: https://arxiv.org/abs/2104.14661 · HTML: https://mlsanity.com/paper/2104.14661/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 42 references that are in the map.
- First posted on arXiv 25 Apr 2021.

## Builds on (most cited first, 11 of 11)

- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing](https://mlsanity.com/paper/2007.06225/) (2020)
- [Do NLP Models Know Numbers? Probing Numeracy in Embeddings](https://mlsanity.com/paper/1909.07940/) (2019)
- [Learning protein sequence embeddings using information from structure](https://mlsanity.com/paper/1902.08661/) (2019)
- [No Training Required: Exploring Random Encoders for Sentence Classification](https://mlsanity.com/paper/1901.10444/) (2019)
- [Is Transfer Learning Necessary for Protein Landscape Prediction?](https://mlsanity.com/paper/2011.03443/) (2020)
- [A Powerful Generative Model Using Random Weights for the Deep Image Representation](https://mlsanity.com/paper/1606.04801/) (2016)
- [On the impressive performance of randomly weighted encoders in summarization tasks](https://mlsanity.com/paper/2002.09084/) (2020)
- [How Powerful Are Randomly Initialized Pointcloud Set Functions?](https://mlsanity.com/paper/2003.05410/) (2020)
- [J-Net: Randomly weighted U-Net for audio source separation](https://mlsanity.com/paper/1911.12926/) (2019)

## Abstract (arXiv)

Large self-supervised models pretrained on millions of protein sequences have recently gained popularity in generating embeddings of protein sequences for protein function prediction. However, the absence of random baselines makes it difficult to conclude whether pretraining has learned useful information for protein function prediction. Here we show that one-hot encoding and random embeddings, both of which do not require any pretraining, are strong baselines for protein function prediction across 14 diverse sequence-to-function tasks.