mlsanityData to 14 Oct 2026

What Topological and Geometric Structure Do Biological Foundation Models Learn? Evidence from 141 Hypotheses

arXiv 2602.22289 ↗First posted 25 Feb 2026q-bio.QM cs.LG q-bio.GNcited by 0

Not on the map. Why?

Abstract (arXiv)

When biological foundation models such as scGPT and Geneformer process single-cell gene expression, what geometric and topological structure forms in their internal representations? Is that structure biologically meaningful or a training artifact, and how confident should we be in such claims? We address these questions through autonomous large-scale hypothesis screening: an AI-driven executor-brainstormer loop that proposed, tested, and refined 141 geometric and topological hypotheses across 52 iterations, covering persistent homology, manifold distances, cross-model alignment, community structure, and directed topology, all with explicit null controls and disjoint gene-pool splits. Three principal findings emerge. First, the models learn genuine geometric structure. Gene embedding neighborhoods exhibit non-trivial topology, with persistent homology significant in 11 of 12 transformer layers at p < 0.05 in the weakest domain and 12 of 12 in the other two. A multi-level distance hierarchy shows that manifold-aware metrics outperform Euclidean distance for identifying regulatory gene pairs, and graph community partitions track known transcription factor target relationships. Second, this structure is shared across independently trained models. CCA alignment between scGPT and Geneformer yields canonical correlation of 0.80 and gene retrieval accuracy of 72 percent, yet none of 19 tested methods reliably recover gene-level correspondences. The models agree on the global shape of gene space but not on precise gene placement. Third, the structure is more localized than it first appears. Under stringent null controls applied across all null families, robust signal concentrates in immune tissue, while lung and external lung signals weaken substantially.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 51 references that are in the map. First posted on arXiv 25 Feb 2026.

Builds on

10 of 51 references.

References · 10

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  3. 2022Toy Models of Superpositioncited by 971
  4. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  5. 2018Computational Optimal Transportcited by 780
  6. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  7. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  8. 2019Ripser: efficient computation of Vietoris-Rips persistence barcodescited by 117
  9. 2026Systematic Evaluation of Single-Cell Foundation Model Interpretability Reveals Attention Captures Co-Expression Rather Than Unique Regulatory Signalcited by 7
  10. 2026Multi-Dimensional Spectral Geometry of Biological Knowledge in Single-Cell Transformer Representationscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FLAG: Foundation model representation with Latent diffusion Alignment via Graph for spatial gene expression prediction
  2. 2026The Geometric Alignment Tax: Tokenization vs. Continuous Geometry in Scientific Foundation Models
  3. 2026Control-Anchored Residual Flow Matching Conditioned on Gene Geometry for Virtual Cell Perturbation Modeling
  4. 2026Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control
  5. 2026Causal Circuit Tracing Reveals Distinct Computational Architectures in Single-Cell Foundation Models: Inhibitory Dominance, Biological Coherence, and Cross-Model Convergence
  6. 2026CoDiffGRN: Rethinking Gene Regulatory Network Inference via the BEELINE-KGC Benchmark and Co-evolutionary Discrete Diffusion
  7. 2026Sparse autoencoders reveal organized biological knowledge but minimal regulatory logic in single-cell foundation models: a comparative atlas of Geneformer and scGPT
  8. 2026Language-encoded network topology enables large language models to reason about complex networks