# Automated Neuron Labelling Enables Generative Steering and Interpretability in Protein Language Models

arXiv: https://arxiv.org/abs/2507.06458 · HTML: https://mlsanity.com/paper/2507.06458/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Interpreting biomedical and science models, one of 13 directions in Mechanistic interpretability.
- Builds on 8 of its 22 references that are in the map.
- First posted on arXiv 8 Jul 2025.

## Placed in

- Mechanistic interpretability / [Interpreting biomedical and science models](https://mlsanity.com/mechinterp/interpreting-biomedical-and-science-models/)

## Builds on (most cited first, 8 of 8)

- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Are Sparse Autoencoders Useful? A Case Study in Sparse Probing](https://mlsanity.com/paper/2502.16681/) (2025)
- [BERTology Meets Biology: Interpreting Attention in Protein Language Models](https://mlsanity.com/paper/2006.15222/) (2020)
- [InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders](https://mlsanity.com/paper/2412.12101/) (2024)
- [Applying sparse autoencoders to unlearn knowledge in language models](https://mlsanity.com/paper/2410.19278/) (2024)
- [Evaluating Open-Source Sparse Autoencoders on Disentangling Factual Knowledge in GPT-2 Small](https://mlsanity.com/paper/2409.04478/) (2024)
- [Interpreting and Steering Protein Language Models through Sparse Autoencoders](https://mlsanity.com/paper/2502.09135/) (2025)

## Built on by (most cited first, 1 of 1)

- [Efficient Auto-Interpretability of AI Models in Biology](https://mlsanity.com/paper/2608.27754/) (2026)

## Abstract (arXiv)

Protein language models (PLMs) encode rich biological information, yet their internal neuron representations are poorly understood. We introduce the first automated framework for labeling every neuron in a PLM with biologically grounded natural language descriptions. Unlike prior approaches relying on sparse autoencoders or manual annotation, our method scales to hundreds of thousands of neurons, revealing individual neurons are selectively sensitive to diverse biochemical and structural properties. We then develop a novel neuron activation-guided steering method to generate proteins with desired traits, enabling convergence to target biochemical properties like molecular weight and instability index as well as secondary and tertiary structural motifs, including alpha helices and canonical Zinc Fingers. We finally show that analysis of labeled neurons in different model sizes reveals PLM scaling laws and a structured neuron space distribution.