mlsanityData to 7 Oct 2026

Causal Circuit Tracing Reveals Distinct Computational Architectures in Single-Cell Foundation Models: Inhibitory Dominance, Biological Coherence, and Cross-Model Convergence

arXiv 2603.01752 ↗First posted 2 Mar 2026cs.LG q-bio.CB q-bio.GNcited by 1

Not on the map. Why?

Abstract (arXiv)

Motivation: Sparse autoencoders (SAEs) decompose foundation model activations into interpretable features, but causal feature-to-feature interactions across network depth remain unknown for biological foundation models. Results: We introduce causal circuit tracing by ablating SAE features and measuring downstream responses, and apply it to Geneformer V2-316M and scGPT whole-human across four conditions (96,892 edges, 80,191 forward passes). Both models show approximately 53 percent biological coherence and 65 to 89 percent inhibitory dominance, invariant to architecture and cell type. scGPT produces stronger effects (mean absolute d = 1.40 vs. 1.05) with more balanced dynamics. Cross-model consensus yields 1,142 conserved domain pairs (10.6x enrichment, p < 0.001). Disease-associated domains are 3.59x more likely to be consensus. Gene-level CRISPRi validation shows 56.4 percent directional accuracy, confirming co-expression rather than causal encoding.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 16 of its 66 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

16 of 66 references.

References · 16

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  6. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  7. 2022Toy Models of Superpositioncited by 928
  8. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  9. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  10. 2024Scaling and evaluating sparse autoencoderscited by 530
  11. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  12. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
4 more
  1. 2021Causal Abstractions of Neural Networkscited by 401
  2. 2013k-Sparse Autoencoderscited by 209
  3. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 183
  4. 2026Systematic Evaluation of Single-Cell Foundation Model Interpretability Reveals Attention Captures Co-Expression Rather Than Unique Regulatory Signalcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.