# One protein is all you need

arXiv: https://arxiv.org/abs/2411.02109 · HTML: https://mlsanity.com/paper/2411.02109/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 117 references that are in the map.
- First posted on arXiv 4 Nov 2024.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling](https://mlsanity.com/paper/1312.3005/) (2013)
- [Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Design](https://mlsanity.com/paper/2402.04997/) (2024)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [Test-Time Training with Masked Autoencoders](https://mlsanity.com/paper/2209.07522/) (2022)
- [ProGen2: Exploring the Boundaries of Protein Language Models](https://mlsanity.com/paper/2206.13517/) (2022)
- [Test-Time Adaptable Neural Networks for Robust Medical Image Segmentation](https://mlsanity.com/paper/2004.04668/) (2020)
- [Diffusion Language Models Are Versatile Protein Learners](https://mlsanity.com/paper/2402.18567/) (2024)
- [Test-Time Training on Nearest Neighbors for Large Language Models](https://mlsanity.com/paper/2305.18466/) (2023)
- [On Pitfalls of Test-Time Adaptation](https://mlsanity.com/paper/2306.03536/) (2023)
- [DPLM-2: A Multimodal Diffusion Protein Language Model](https://mlsanity.com/paper/2410.13782/) (2024)
- [PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains](https://mlsanity.com/paper/2102.12206/) (2021)
- [Ankh: Optimized Protein Language Model Unlocks General-Purpose Modelling](https://mlsanity.com/paper/2301.06568/) (2023)
- [Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs](https://mlsanity.com/paper/2410.08020/) (2024)
- [Test-Time Training on Video Streams](https://mlsanity.com/paper/2307.05014/) (2023)
- [Learning to Generalize across Domains on Single Test Samples](https://mlsanity.com/paper/2202.08045/) (2022)
- [Adapting to Distribution Shift by Visual Domain Prompt Generation](https://mlsanity.com/paper/2405.02797/) (2024)
- [Self-Supervised Test-Time Learning for Reading Comprehension](https://mlsanity.com/paper/2103.11263/) (2021)
- [Improving Protein Optimization with Smoothed Fitness Landscapes](https://mlsanity.com/paper/2307.00494/) (2023)
- [Learning to design protein-protein interactions with enhanced generalization](https://mlsanity.com/paper/2310.18515/) (2023)
- [HelixFold-Single: MSA-free Protein Structure Prediction by Using Protein Language Model as an Alternative](https://mlsanity.com/paper/2207.13921/) (2022)
- [Deriving Language Models from Masked Language Models](https://mlsanity.com/paper/2305.15501/) (2023)
- [Deep Reinforcement Learning for Modelling Protein Complexes](https://mlsanity.com/paper/2405.02299/) (2024)
- [Antibody DomainBed: Out-of-Distribution Generalization in Therapeutic Protein Design](https://mlsanity.com/paper/2407.21028/) (2024)

## Abstract (arXiv)

Generalization beyond training data remains a central challenge in machine learning for biology. A common way to enhance generalization is self-supervised pre-training on large datasets. However, aiming to perform well on all possible proteins can limit a model's capacity to excel on any specific one, whereas experimentalists typically need accurate predictions for individual proteins they study, often not covered in training data. To address this limitation, we propose a method that enables self-supervised customization of protein language models to one target protein at a time, on the fly, and without assuming any additional data. We show that our Protein Test-Time Training (ProteinTTT) method consistently enhances generalization across different models, their sizes, and datasets. ProteinTTT improves structure prediction for challenging targets, achieves new state-of-the-art results on protein fitness prediction, and enhances function prediction on two tasks. Through two challenging case studies, we also show that customization via ProteinTTT achieves more accurate antibody-antigen loop modeling and enhances 19% of structures in the Big Fantastic Virus Database, delivering improved predictions where general-purpose AlphaFold2 and ESMFold struggle.