# Systematic Evaluation of Single-Cell Foundation Model Interpretability Reveals Attention Captures Co-Expression Rather Than Unique Regulatory Signal

arXiv: https://arxiv.org/abs/2602.17532 · HTML: https://mlsanity.com/paper/2602.17532/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Interpreting biomedical and science models, one of 13 directions in Mechanistic interpretability.
- First posted on arXiv 19 Feb 2026.

## Placed in

- Mechanistic interpretability / [Interpreting biomedical and science models](https://mlsanity.com/mechinterp/interpreting-biomedical-and-science-models/)

## Built on by (most cited first, 7 of 7)

- [Sparse autoencoders reveal organized biological knowledge but minimal regulatory logic in single-cell foundation models: a comparative atlas of Geneformer and scGPT](https://mlsanity.com/paper/2603.02952/) (2026)
- [From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discovery](https://mlsanity.com/paper/2607.12474/) (2026)
- [Multi-Dimensional Spectral Geometry of Biological Knowledge in Single-Cell Transformer Representations](https://mlsanity.com/paper/2602.22247/) (2026)
- [Causal Circuit Tracing Reveals Distinct Computational Architectures in Single-Cell Foundation Models: Inhibitory Dominance, Biological Coherence, and Cross-Model Convergence](https://mlsanity.com/paper/2603.01752/) (2026)
- [Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control](https://mlsanity.com/paper/2603.11940/) (2026)
- [What Topological and Geometric Structure Do Biological Foundation Models Learn? Evidence from 141 Hypotheses](https://mlsanity.com/paper/2602.22289/) (2026)
- [Discovery of a Hematopoietic Manifold in scGPT Yields a Method for Extracting Performant Algorithms from Biological Foundation Model Internals](https://mlsanity.com/paper/2603.10261/) (2026)

## Abstract (arXiv)

We present a systematic evaluation framework - thirty-seven analyses, 153 statistical tests, four cell types, two perturbation modalities - for assessing mechanistic interpretability in single-cell foundation models. Applying this framework to scGPT and Geneformer, we find that attention patterns encode structured biological information with layer-specific organisation - protein-protein interactions in early layers, transcriptional regulation in late layers - but this structure provides no incremental value for perturbation prediction: trivial gene-level baselines outperform both attention and correlation edges (AUROC 0.81-0.88 versus 0.70), pairwise edge scores add zero predictive contribution, and causal ablation of regulatory heads produces no degradation. These findings generalise from K562 to RPE1 cells; the attention-correlation relationship is context-dependent, but gene-level dominance is universal. Cell-State Stratified Interpretability (CSSI) addresses an attention-specific scaling failure, improving GRN recovery up to 1.85x. The framework establishes reusable quality-control standards for the field.