# Multi-Scale Representation Learning for Protein Fitness Prediction

arXiv: https://arxiv.org/abs/2412.01108 · HTML: https://mlsanity.com/paper/2412.01108/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2024.

## Built on by (most cited first, 11 of 11)

- [AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation](https://mlsanity.com/paper/2605.28655/) (2026)
- [Steering Protein Language Models](https://mlsanity.com/paper/2509.07983/) (2025)
- [Understanding protein function with a multimodal retrieval-augmented foundation model](https://mlsanity.com/paper/2508.04724/) (2025)
- [Rank-and-Reason: Multi-Agent Collaboration Accelerates Zero-Shot Protein Mutation Prediction](https://mlsanity.com/paper/2602.00197/) (2026)
- [NABench: Large-Scale Benchmarks of Nucleotide Foundation Models for Fitness Prediction](https://mlsanity.com/paper/2511.02888/) (2025)
- [InstructPLM-mu: 1-Hour Fine-Tuning of ESM2 Beats ESM3 in Protein Mutation Predictions](https://mlsanity.com/paper/2510.03370/) (2025)
- [Evolutionary Profiles for Protein Fitness Prediction](https://mlsanity.com/paper/2510.07286/) (2025)
- [Few-shot Protein Fitness Prediction via In-context Learning and Test-time Training](https://mlsanity.com/paper/2512.02315/) (2025)
- [Towards Multiscale Graph-based Protein Learning with Geometric Secondary Structural Motifs](https://mlsanity.com/paper/2602.00862/) (2026)
- [PFArena: Benchmarking Language Models for Protein Modification](https://mlsanity.com/paper/2609.28921/) (2026)
- [Structure-aware Reinforcement Learning for Protein Directed Evolution](https://mlsanity.com/paper/2609.39048/) (2026)

## Abstract (arXiv)

Designing novel functional proteins crucially depends on accurately modeling their fitness landscape. Given the limited availability of functional annotations from wet-lab experiments, previous methods have primarily relied on self-supervised models trained on vast, unlabeled protein sequence or structure datasets. While initial protein representation learning studies solely focused on either sequence or structural features, recent hybrid architectures have sought to merge these modalities to harness their respective strengths. However, these sequence-structure models have so far achieved only incremental improvements when compared to the leading sequence-only approaches, highlighting unresolved challenges effectively leveraging these modalities together. Moreover, the function of certain proteins is highly dependent on the granular aspects of their surface topology, which have been overlooked by prior models. To address these limitations, we introduce the Sequence-Structure-Surface Fitness (S3F) model - a novel multimodal representation learning framework that integrates protein features across several scales. Our approach combines sequence representations from a protein language model with Geometric Vector Perceptron networks encoding protein backbone and detailed surface topology. The proposed method achieves state-of-the-art fitness prediction on the ProteinGym benchmark encompassing 217 substitution deep mutational scanning assays, and provides insights into the determinants of protein function. Our code is at https://github.com/DeepGraphLearning/S3F.