mlsanityData to 14 Oct 2026

SAE-RNA: A Sparse Autoencoder Model for Interpreting RNA Language Model Representations

arXiv 2510.02734 ↗First posted 3 Oct 2025q-bio.BM cs.AI q-bio.GNcited by 0

On the map: Mechanistic interpretability → Interpreting biomedical and science models

Abstract (arXiv)

Deep learning, particularly with the advancement of Large Language Models, has transformed biomolecular modeling, with protein language models such as ESM inspiring emerging RNA language models such as RiNALMo. Recent work has begun applying sparse autoencoders (SAEs) to protein language model representations, exploring representation-level interpretability in biomolecular models. Here, we explore whether SAEs can provide interpretable feature decompositions of RNA language model representations, while also examining their limitations in this setting. We present SAE-RNA, interpretability model that analyzes RiNALMo representations and maps them to known human-level biological features. Rather than claiming definitive biological concept discovery, our study frames SAE-based analysis as a representation-level probe for characterizing how RNA language models organize biological information internally. More broadly, SAE-RNA provides a feature-level framework for comparing RNA groups and identifying sparse representation components associated with RNA family identity or structural context.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpreting biomedical and science models, one of 13 directions in Mechanistic interpretability. Builds on 5 of its 15 references that are in the map. First posted on arXiv 3 Oct 2025.

Builds on

5 of 15 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders13 Nov

Other references · 4

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  2. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  4. 2024RiNALMo: General-Purpose RNA Language Models Can Generalize Well on Structure Prediction Taskscited by 33

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ProtSAE: Disentangling and Interpreting Protein Language Models via Semantically-Guided Sparse Autoencoders
  2. 2025Interpreting and Steering Protein Language Models through Sparse Autoencoders
  3. 2024Can sparse autoencoders make sense of gene expression latent variable models?
  4. 2025Towards Interpretable Protein Structure Prediction with Sparse Autoencoders
  5. 2026Interpreting Latent Protein Language Model Features with Geometric Annotations
  6. 2025Mechanistic Interpretability of Antibody Language Models Using SAEs
  7. 2026Sparse autoencoders reveal organized biological knowledge but minimal regulatory logic in single-cell foundation models: a comparative atlas of Geneformer and scGPT
  8. 2025Sparse Autoencoders for Low-N Protein Function Prediction and Design