# Protein Circuit Tracing via Cross-layer Transcoders

arXiv: https://arxiv.org/abs/2602.12026 · HTML: https://mlsanity.com/paper/2602.12026/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Interpreting biomedical and science models, one of 13 directions in Mechanistic interpretability.
- Builds on 13 of its 56 references that are in the map.
- First posted on arXiv 12 Feb 2026.

## Placed in

- Mechanistic interpretability / [Interpreting biomedical and science models](https://mlsanity.com/mechinterp/interpreting-biomedical-and-science-models/)

## Builds on (most cited first, 13 of 13)

- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Scaling and evaluating sparse autoencoders](https://mlsanity.com/paper/2406.04093/) (2024)
- [k-Sparse Autoencoders](https://mlsanity.com/paper/1312.5663/) (2013)
- [Transcoders Find Interpretable LLM Feature Circuits](https://mlsanity.com/paper/2406.11944/) (2024)
- [InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders](https://mlsanity.com/paper/2412.12101/) (2024)
- [Improving Protein Optimization with Smoothed Fitness Landscapes](https://mlsanity.com/paper/2307.00494/) (2023)
- [Interpreting and Steering Protein Language Models through Sparse Autoencoders](https://mlsanity.com/paper/2502.09135/) (2025)
- [Towards Interpretable Protein Structure Prediction with Sparse Autoencoders](https://mlsanity.com/paper/2503.08764/) (2025)
- [On Recovering Higher-order Interactions from Protein Language Models](https://mlsanity.com/paper/2405.06645/) (2024)
- [Steering Protein Language Models](https://mlsanity.com/paper/2509.07983/) (2025)
- [SHAP zero Explains Biological Sequence Models with Near-zero Marginal Cost for Future Queries](https://mlsanity.com/paper/2410.19236/) (2024)
- [Sparse Autoencoders for Low-N Protein Function Prediction and Design](https://mlsanity.com/paper/2508.18567/) (2025)
- [SpecMER: Fast Protein Generation with K-mer Guided Speculative Decoding](https://mlsanity.com/paper/2509.21689/) (2025)

## Built on by (most cited first, 3 of 3)

- [What Does a Chemical Language Model Know About Molecules?](https://mlsanity.com/paper/2606.23443/) (2026)
- [Retrieval and competition: how a protein foundation model starts a protein](https://mlsanity.com/paper/2605.16331/) (2026)
- [Circuit Tracing in Autoregressive Protein Language Models](https://mlsanity.com/paper/2606.16044/) (2026)

## Abstract (arXiv)

Protein language models (pLMs) have emerged as powerful predictors of protein structure and function. However, the computational circuits underlying their predictions remain poorly understood. Recent mechanistic interpretability methods decompose pLM representations into interpretable features, but they treat each layer independently and thus fail to capture cross-layer computation, limiting their ability to approximate the full model. We introduce ProtoMech, a framework for discovering computational circuits in pLMs using cross-layer transcoders that learn sparse latent representations jointly across layers to capture the model's full computational circuitry. Applied to the pLM ESM2, ProtoMech recovers 82-89% of the original performance on protein family classification and function prediction tasks. ProtoMech then identifies compressed circuits that use <1% of the latent space while retaining up to 79% of model accuracy, revealing correspondence with structural and functional motifs, including binding, signaling, and stability. Steering along these circuits enables high-fitness protein design, surpassing baseline methods in more than 70% of cases. These results establish ProtoMech as a principled framework for protein circuit tracing.