# Discovering Functionally Selective Brain Regions with a Deep Topographic Multimodal Model

arXiv: https://arxiv.org/abs/2606.09770 · HTML: https://mlsanity.com/paper/2606.09770/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 64 references that are in the map.
- First posted on arXiv 8 Jun 2026.

## Builds on (most cited first, 19 of 19)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content](https://mlsanity.com/paper/2410.08260/) (2024)
- [OmniBench: Towards The Future of Universal Omni-Language Models](https://mlsanity.com/paper/2409.15272/) (2024)
- [Machine Learning for Neuroimaging with Scikit-Learn](https://mlsanity.com/paper/1412.3919/) (2014)
- [The neuroconnectionist research programme](https://mlsanity.com/paper/2209.03718/) (2022)
- [The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant Units](https://mlsanity.com/paper/2411.02280/) (2024)
- [From Language to Cognition: How LLMs Outgrow the Human Language Network](https://mlsanity.com/paper/2503.01830/) (2025)
- [TRIBE: TRImodal Brain Encoder for whole-brain fMRI response prediction](https://mlsanity.com/paper/2507.22229/) (2025)
- [Omni-Embed-Nemotron: A Unified Multimodal Retrieval Model for Text, Image, Audio, and Video](https://mlsanity.com/paper/2510.03458/) (2025)
- [Scaling Laws for Task-Optimized Models of the Primate Visual Ventral Stream](https://mlsanity.com/paper/2411.05712/) (2024)
- [TopoLM: brain-like spatio-functional organization in a topographic language model](https://mlsanity.com/paper/2410.11516/) (2024)
- [Modeling Category-Selective Cortical Regions with Topographic Variational Autoencoders](https://mlsanity.com/paper/2110.13911/) (2021)
- [TopoNets: High Performing Vision and Language Models with Brain-Like Topography](https://mlsanity.com/paper/2501.16396/) (2025)
- [End-to-end topographic networks as models of cortical map formation and human visual behaviour: moving beyond convolutions](https://mlsanity.com/paper/2308.09431/) (2023)
- [VIBE: Video-Input Brain Encoder for fMRI Response Modeling](https://mlsanity.com/paper/2507.17958/) (2025)
- [Predicting Brain Responses To Natural Movies With Multimodal LLMs](https://mlsanity.com/paper/2507.19956/) (2025)
- [Alignment between Brains and AI: Evidence for Convergent Evolution across Modalities, Scales and Training Trajectories](https://mlsanity.com/paper/2507.01966/) (2025)

## Built on by (most cited first, 2 of 2)

- [Modular Cognitive Architecture Emerges in Large Language Models](https://mlsanity.com/paper/2608.13567/) (2026)
- [The Platonic brain bridge hypothesis: human brain networks as an architectural prior for multimodal large language models](https://mlsanity.com/paper/2609.10947/) (2026)

## Abstract (arXiv)

Nearby neurons in cortex share similar response profiles, producing systematic spatial organization across sensory and cognitive systems. Recent topographic models reproduce aspects of this structure but remain unimodal and spatially constrain each layer separately, yielding fragmented maps that capture neither the contiguity of cortical processing streams nor their integration across modalities. We introduce Topo-Omni, a topographic multimodal model in which visual, auditory, and language/cognitive processing share a single contiguous in-silico sheet. Built by fine-tuning a pretrained foundation model with a spatial smoothness objective, this architecture develops clusters across modalities that are consistent with human neuroimaging, from sensory to cognitive systems. Driving or suppressing a cluster selectively biases or impairs perception, paralleling human intervention studies. Finally, we use our model to screen for novel clusters in-silico and discover new natural landscape and animal networks which we validate in human data. A single spatial principle thus organizes representations across modalities and processing stages, yielding testable hypotheses about cortical organization.