# Manifolds and Modules: How Function Develops in a Neural Foundation Model

arXiv: https://arxiv.org/abs/2512.07869 · HTML: https://mlsanity.com/paper/2512.07869/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 59 references that are in the map.
- First posted on arXiv 26 Nov 2025.

## Builds on (most cited first, 12 of 12)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Neural population geometry: An approach for understanding biological and artificial neural networks](https://mlsanity.com/paper/2104.07059/) (2021)
- [The neuroconnectionist research programme](https://mlsanity.com/paper/2209.03718/) (2022)
- [Neural system identification for large populations separating "what" and "where"](https://mlsanity.com/paper/1711.02653/) (2017)
- [V1T: large-scale mouse V1 response prediction using a Vision Transformer](https://mlsanity.com/paper/2302.03023/) (2023)
- [Deep Spiking Neural Networks with High Representation Similarity Model Visual Pathways of Macaque and Mouse](https://mlsanity.com/paper/2303.06060/) (2023)
- [IAN: Iterated Adaptive Neighborhoods for manifold learning and dimensionality estimation](https://mlsanity.com/paper/2208.09123/) (2022)
- [Reproducibility of predictive networks for mouse visual cortex](https://mlsanity.com/paper/2406.12625/) (2024)
- [Mice to Machines: Neural Representations from Visual Cortex for Domain Generalization](https://mlsanity.com/paper/2505.06886/) (2025)
- [Learning to cluster neuronal function](https://mlsanity.com/paper/2506.03293/) (2025)

## Abstract (arXiv)

Foundation models have shown remarkable success in fitting biological visual systems; however, their black-box nature inherently limits their utility for understanding brain function. Here, we peek inside a SOTA foundation model of neural activity (Wang et al., 2025) as a physiologist might, characterizing each 'neuron' based on its temporal response properties to parametric stimuli. We analyze how different stimuli are represented in neural activity space by building decoding manifolds, and we analyze how different neurons are represented in stimulus-response space by building neural encoding manifolds. We find that the different processing stages of the model (i.e., the feedforward encoder, recurrent, and readout modules) each exhibit qualitatively different representational structures in these manifolds. The recurrent module shows a jump in capabilities over the encoder module by 'pushing apart' the representations of different temporal stimulus patterns; while the readout module achieves biological fidelity by using numerous specialized feature maps rather than biologically plausible mechanisms. Overall, we present this work as a study of the inner workings of a prominent neural foundation model, gaining insights into the biological relevance of its internals through the novel analysis of its neurons' joint temporal response patterns.