mlsanityData to 14 Oct 2026

InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders

arXiv 2412.12101 ↗First posted 13 Nov 2024q-bio.BM cs.AI cs.LGcited by 66

On the map: Mechanistic interpretability → Interpreting biomedical and science models

Abstract (arXiv)

Protein language models (PLMs) have demonstrated remarkable success in protein modeling and design, yet their internal mechanisms for predicting structure and function remain poorly understood. Here we present a systematic approach to extract and analyze interpretable features from PLMs using sparse autoencoders (SAEs). By training SAEs on embeddings from the PLM ESM-2, we identify up to 2,548 human-interpretable latent features per layer that strongly correlate with up to 143 known biological concepts such as binding sites, structural motifs, and functional domains. In contrast, examining individual neurons in ESM-2 reveals up to 46 neurons per layer with clear conceptual alignment across 15 known concepts, suggesting that PLMs represent most concepts in superposition. Beyond capturing known annotations, we show that ESM-2 learns coherent concepts that do not map onto existing annotations and propose a pipeline using language models to automatically interpret novel latent features learned by the SAEs. As practical applications, we demonstrate how these latent features can fill in missing annotations in protein databases and enable targeted steering of protein sequence generation. Our results demonstrate that PLMs encode rich, interpretable representations of protein biology and we propose a systematic framework to extract and analyze these latent features. In the process, we recover both known biology and potentially new protein motifs. As community resources, we introduce InterPLM (interPLM.ai), an interactive visualization platform for exploring and analyzing learned PLM features, and release code for training and analysis at github.com/ElanaPearl/interPLM.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). Placed in Interpreting biomedical and science models, one of 13 directions in Mechanistic interpretability. Builds on 9 of its 30 references that are in the map. First posted on arXiv 13 Nov 2024.

Builds on

9 of 30 references.

References · 9

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  3. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  4. 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
  5. 2022ProGen2: Exploring the Boundaries of Protein Language Modelscited by 159
  6. 2021Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factorscited by 108
  7. 2024Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Modelscited by 51
  8. 2024NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internalscited by 45
  9. 2023Insights Into the Inner Workings of Transformer Models for Protein Function Predictioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 31; 2026: 34.

66 papers.

Sort:

By area:

  1. 2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometrycited by 61
  2. 2025Understanding the Repeat Curse in Large Language Models from a Feature Perspectivecited by 37
  3. 2025Position: Use Sparse Autoencoders to Discover Unknownscited by 19
  4. 2025Protein Large Language Models: A Comprehensive Surveycited by 18
  5. 2025From superposition to sparse codes: interpretable representations in neural networkscited by 16
  6. 2025Towards Mechanistic Interpretability of Graph Transformers via Attention Graphscited by 13
  7. 2025Discovering and Steering Interpretable Concepts in Large Generative Music Modelscited by 13
  8. 2025Towards Interpretable Protein Structure Prediction with Sparse Autoencoderscited by 12
  9. 2025Towards mechanistic understanding in a data-driven weather model: internal activations reveal interpretable physical featurescited by 12
  10. 2025AbsTopK: Rethinking Sparse Autoencoders For Bidirectional Featurescited by 9
  11. 2026AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoderscited by 8
  12. 2025Graph-Aware Isomorphic Attention for Adaptive Dynamics in Transformerscited by 7
38 more
  1. 2025Computational Protein Science in the Era of Large Language Models (LLMs)cited by 6
  2. 2025ProteinZero: Self-Improving Protein Generation via Online Reinforcement Learningcited by 6
  3. 2026Controlling Repetition in Protein Language Modelscited by 5
  4. 2026Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoderscited by 5
  5. 2025Sparse Autoencoders for Low-N Protein Function Prediction and Designcited by 4
  6. 2026From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discoverycited by 4
  7. 2025Resurrecting the Salmon: Rethinking Mechanistic Interpretability with Domain-Specific Sparse Autoencoderscited by 3
  8. 2025ProtSAE: Disentangling and Interpreting Protein Language Models via Semantically-Guided Sparse Autoencoderscited by 3
  9. 2025A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious Minimacited by 3
  10. 2026Protein Circuit Tracing via Cross-layer Transcoderscited by 3
  11. 2024Scaling Structure Aware Virtual Screening to Billions of Molecules with SPRINTcited by 2
  12. 2025Insights into a radiology-specialised multimodal large language model with sparse autoencoderscited by 2
  13. 2025Towards Understanding the Shape of Representations in Protein Language Modelscited by 2
  14. 2025CXR-LanIC: Language-Grounded Interpretable Classifier for Chest X-Ray Diagnosiscited by 2
  15. 2025Discovering Interpretable Biological Concepts in Single-cell RNA-seq Foundation Modelscited by 2
  16. 2025Unveiling Latent Knowledge in Chemistry Language Models through Sparse Autoencoderscited by 2
  17. 2025Circuits, Features, and Heuristics in Molecular Transformerscited by 2
  18. 2026Two Stages of Folding: Convergent Mechanisms in AI Protein Folding Trunkscited by 2
  19. 2026Protein Language Models Diverge from Natural Language: Comparative Analysis and Improved Inferencecited by 2
  20. 2026Inference-Time Toxicity Mitigation in Protein Language Modelscited by 2
  21. 2026On the Relationship Between Activation Outliers and Feature Death in Sparse Autoencoderscited by 2
  22. 2026What Does a Chemical Language Model Know About Molecules?cited by 2
  23. 2026Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligencecited by 2
  24. 2025PLM-eXplain: Divide and Conquer the Protein Embedding Spacecited by 1
  25. 2025Automated Neuron Labelling Enables Generative Steering and Interpretability in Protein Language Modelscited by 1
  26. 2026Induction Meets Biology: Mechanisms of Repeat Detection in Protein Language Modelscited by 1
  27. 2026Retrieval and competition: how a protein foundation model starts a proteincited by 1
  28. 2026A Geometric View for Understanding Concept Learning and Neuron Interpretation in Sparse Autoencoderscited by 1
  29. 2026VFUSE: Virulent Feature Understanding with Sparse autoEncoderscited by 1
  30. 2026Mechanistic Interpretability and Causal Feature Steering of Neural Quantum States via Sparse Autoencoderscited by 1
  31. 2026Probing and steering biology across Boltz-1s trunk-diffusion boundarycited by 1
  32. 2025GPT and Prejudice: A Sparse Approach to Understanding Learned Representations in Large Language Modelscited by 0
  33. 2025BioBlobs: Unsupervised Discovery of Functional Substructures for Protein Function Predictioncited by 0
  34. 2025SAE-RNA: A Sparse Autoencoder Model for Interpreting RNA Language Model Representationscited by 0
  35. 2025Equivariant Sparse Autoencoders: Mechanistic Interpretability of Neural Networks on Symmetric Datacited by 0
  36. 2025Quantifying the Role of OpenFold Components in Protein Structure Predictioncited by 0
  37. 2025Mechanistic Interpretability of Antibody Language Models Using SAEscited by 0
  38. 2026No Generation without Representation: Efficient Causal Protein Language Models Enable Zero-Shot Fitness Estimationcited by 0

50 most cited of 66 All 66 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Interpreting and Steering Protein Language Models through Sparse Autoencoders
  2. 2026Structural Interpretations of Protein Language Model Representations via Differentiable Graph Partitioning
  3. 2026Learning Latent Protein Languages for Autoregressive Generation
  4. 2025Structure-Aligned Protein Language Model
  5. 2024TourSynbio: A Multi-Modal Large Model and Agent Framework to Bridge Text and Protein Sequences for Protein Engineering
  6. 2023Identification of Knowledge Neurons in Protein Language Models
  7. 2023InstructProtein: Aligning Human and Protein Language via Knowledge Instruction
  8. 2026Generative modeling of intrinsically disordered protein regions by reinforcing sparse autoencoder features