One protein is all you need
Not on the map. Why?
Abstract (arXiv)
Generalization beyond training data remains a central challenge in machine learning for biology. A common way to enhance generalization is self-supervised pre-training on large datasets. However, aiming to perform well on all possible proteins can limit a model's capacity to excel on any specific one, whereas experimentalists typically need accurate predictions for individual proteins they study, often not covered in training data. To address this limitation, we propose a method that enables self-supervised customization of protein language models to one target protein at a time, on the fly, and without assuming any additional data. We show that our Protein Test-Time Training (ProteinTTT) method consistently enhances generalization across different models, their sizes, and datasets. ProteinTTT improves structure prediction for challenging targets, achieves new state-of-the-art results on protein fitness prediction, and enhances function prediction on two tasks. Through two challenging case studies, we also show that customization via ProteinTTT achieves more accurate antibody-antigen loop modeling and enhances 19% of structures in the Big Fantastic Virus Database, delivering improved predictions where general-purpose AlphaFold2 and ESMFold struggle.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 117 references that are in the map. First posted on arXiv 4 Nov 2024.
Builds on
29 of 117 references.
References · 29
- 2017Attention Is All You Needcited by 60,361
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2021Zero-Shot Text-to-Image Generationcited by 4,288
- 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
- 2016An overview of gradient descent optimization algorithmscited by 1,033
- 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 308
- 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 305
- 2019Evaluating Protein Transfer Learning with TAPEcited by 231
- 2022Test-Time Training with Masked Autoencoderscited by 212
- 2022ProGen2: Exploring the Boundaries of Protein Language Modelscited by 159
- 2020Test-Time Adaptable Neural Networks for Robust Medical Image Segmentationcited by 102
17 more
- 2024Diffusion Language Models Are Versatile Protein Learnerscited by 98
- 2023Test-Time Training on Nearest Neighbors for Large Language Modelscited by 88
- 2023On Pitfalls of Test-Time Adaptationcited by 64
- 2024DPLM-2: A Multimodal Diffusion Protein Language Modelcited by 64
- 2021PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domainscited by 61
- 2023Ankh: Optimized Protein Language Model Unlocks General-Purpose Modellingcited by 45
- 2024Efficiently Learning at Test-Time: Active Fine-Tuning of LLMscited by 42
- 2023Test-Time Training on Video Streamscited by 37
- 2022Learning to Generalize across Domains on Single Test Samplescited by 24
- 2024Adapting to Distribution Shift by Visual Domain Prompt Generationcited by 23
- 2021Self-Supervised Test-Time Learning for Reading Comprehensioncited by 22
- 2023Improving Protein Optimization with Smoothed Fitness Landscapescited by 21
- 2023Learning to design protein-protein interactions with enhanced generalizationcited by 16
- 2022HelixFold-Single: MSA-free Protein Structure Prediction by Using Protein Language Model as an Alternativecited by 15
- 2023Deriving Language Models from Masked Language Modelscited by 9
- 2024Deep Reinforcement Learning for Modelling Protein Complexescited by 5
- 2024Antibody DomainBed: Out-of-Distribution Generalization in Therapeutic Protein Designcited by 3
Built on by
Not cited yet.