# Sparse Autoencoders for Low-N Protein Function Prediction and Design

arXiv: https://arxiv.org/abs/2508.18567 · HTML: https://mlsanity.com/paper/2508.18567/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Interpreting biomedical and science models, one of 13 directions in Mechanistic interpretability.
- Builds on 7 of its 35 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Placed in

- Mechanistic interpretability / [Interpreting biomedical and science models](https://mlsanity.com/mechinterp/interpreting-biomedical-and-science-models/)

## Builds on (most cited first, 7 of 7)

- [Scaling and evaluating sparse autoencoders](https://mlsanity.com/paper/2406.04093/) (2024)
- [Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval](https://mlsanity.com/paper/2205.13760/) (2022)
- [InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders](https://mlsanity.com/paper/2412.12101/) (2024)
- [Interpreting and Steering Protein Language Models through Sparse Autoencoders](https://mlsanity.com/paper/2502.09135/) (2025)
- [Towards Interpretable Protein Structure Prediction with Sparse Autoencoders](https://mlsanity.com/paper/2503.08764/) (2025)
- [On Recovering Higher-order Interactions from Protein Language Models](https://mlsanity.com/paper/2405.06645/) (2024)
- [SHAP zero Explains Biological Sequence Models with Near-zero Marginal Cost for Future Queries](https://mlsanity.com/paper/2410.19236/) (2024)

## Built on by (most cited first, 4 of 4)

- [Protein Circuit Tracing via Cross-layer Transcoders](https://mlsanity.com/paper/2602.12026/) (2026)
- [Structure-Aware Masking for Protein Representation Learning](https://mlsanity.com/paper/2605.16581/) (2026)
- [Circuit Tracing in Autoregressive Protein Language Models](https://mlsanity.com/paper/2606.16044/) (2026)
- [Learning Task-Specific Antibody Representations via Function-Aware Masking](https://mlsanity.com/paper/2609.00518/) (2026)

## Abstract (arXiv)

Predicting protein function from amino acid sequence remains a central challenge in data-scarce (low-N) regimes, limiting machine learning-guided protein design when only small amounts of assay-labeled sequence-function data are available. Protein language models (pLMs) have advanced the field by providing evolutionary-informed embeddings and sparse autoencoders (SAEs) have enabled decomposition of these embeddings into interpretable latent variables that capture structural and functional features. However, the effectiveness of SAEs for low-N function prediction and protein design has not been systematically studied. Herein, we evaluate SAEs trained on fine-tuned ESM2 embeddings across diverse fitness extrapolation and protein engineering tasks. We show that SAEs, with as few as 24 sequences, consistently outperform or compete with their ESM2 baselines in fitness prediction, indicating that their sparse latent space encodes compact and biologically meaningful representations that generalize more effectively from limited data. Moreover, steering predictive latents exploits biological motifs in pLM representations, yielding top-fitness variants in 83% of cases compared to designing with ESM2 alone.