# OmniMouse: Scaling properties of multi-modal, multi-task Brain Models on 150B Neural Tokens

arXiv: https://arxiv.org/abs/2604.18827 · HTML: https://mlsanity.com/paper/2604.18827/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 94 references that are in the map.
- First posted on arXiv 20 Apr 2026.

## Builds on (most cited first, 23 of 23)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies](https://mlsanity.com/paper/2404.06395/) (2024)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI](https://mlsanity.com/paper/2405.18765/) (2024)
- [BrainBERT: Self-supervised representation learning for intracranial recordings](https://mlsanity.com/paper/2302.14367/) (2023)
- [Brain decoding: toward real-time reconstruction of visual perception](https://mlsanity.com/paper/2310.19812/) (2023)
- [EEGFormer: Towards Transferable and Interpretable Large-Scale EEG Foundation Model](https://mlsanity.com/paper/2401.10278/) (2024)
- [Representation learning for neural population activity with Neural Data Transformers](https://mlsanity.com/paper/2108.01210/) (2021)
- [SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals](https://mlsanity.com/paper/2405.17766/) (2024)
- [Neural Encoding and Decoding at Scale](https://mlsanity.com/paper/2504.08201/) (2025)
- [TRIBE: TRImodal Brain Encoder for whole-brain fMRI response prediction](https://mlsanity.com/paper/2507.22229/) (2025)
- [A rotation-equivariant convolutional neural network model of primary visual cortex](https://mlsanity.com/paper/1809.10504/) (2018)
- [The Sensorium competition on predicting large-scale mouse primary visual cortex activity](https://mlsanity.com/paper/2206.08666/) (2022)
- [Scaling Laws for Task-Optimized Models of the Primate Visual Ventral Stream](https://mlsanity.com/paper/2411.05712/) (2024)
- [Neuroformer: Multimodal and Multitask Generative Pretraining for Brain Data](https://mlsanity.com/paper/2311.00136/) (2023)
- [POCO: Scalable Neural Forecasting through Population Conditioning](https://mlsanity.com/paper/2506.14957/) (2025)
- [V1T: large-scale mouse V1 response prediction using a Vision Transformer](https://mlsanity.com/paper/2302.03023/) (2023)
- [Modeling the Human Visual System: Comparative Insights from Response-Optimized and Task-Optimized Vision Models, Language Models, and different Readout Mechanisms](https://mlsanity.com/paper/2410.14031/) (2024)
- [GPT2MEG: Quantizing MEG for Autoregressive Generation](https://mlsanity.com/paper/2404.09256/) (2024)
- [Modeling Dynamic Neural Activity by combining Naturalistic Video Stimuli and Stimulus-independent Latent Factors](https://mlsanity.com/paper/2410.16136/) (2024)
- [QuantFormer: Learning to Quantize for Neural Activity Forecasting in Mouse Visual Cortex](https://mlsanity.com/paper/2412.07264/) (2024)

## Built on by (most cited first, 4 of 4)

- [NEvo: Neural-Guided Evolutionary Video Synthesis for Dynamic Visual Selectivity](https://mlsanity.com/paper/2607.02317/) (2026)
- [Leveraging unlabelled data for generalizable neural population decoding](https://mlsanity.com/paper/2607.14086/) (2026)
- [CAPT: A Multi-task Continuous Autoregressive Transformer enabling Cross-dataset and Cross-species Transfer for Calcium Population Dynamics](https://mlsanity.com/paper/2607.23258/) (2026)
- [Language as the Interface: Foundation-Model Contrastive Learning Links Transcriptomes and Electrophysiology](https://mlsanity.com/paper/2609.37024/) (2026)

## Abstract (arXiv)

Scaling data and artificial neural networks has transformed AI, driving breakthroughs in language and vision. Whether similar principles apply to modeling brain activity remains unclear. Here we leveraged a dataset of 3.1 million neurons from the visual cortex of 73 mice across 323 sessions, totaling more than 150 billion neural tokens recorded during natural movies, images and parametric stimuli, and behavior. We train multi-modal, multi-task models that support three regimes flexibly at test time: neural prediction, behavioral decoding, neural forecasting, or any combination of the three. OmniMouse achieves state-of-the-art performance, outperforming specialized baselines across nearly all evaluation regimes. We find that performance scales reliably with more data, but gains from increasing model size saturate. This inverts the standard AI scaling story: in language and computer vision, massive datasets make parameter scaling the primary driver of progress, whereas in brain modeling -- even in the mouse visual cortex, a relatively simple system -- models remain data-limited despite vast recordings. The observation of systematic scaling raises the possibility of phase transitions in neural modeling, where larger and richer datasets might unlock qualitatively new capabilities, paralleling the emergent properties seen in large language models. Code available at https://github.com/enigma-brain/omnimouse.