# Scaling laws for decoding images from brain activity

arXiv: https://arxiv.org/abs/2501.15322 · HTML: https://mlsanity.com/paper/2501.15322/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 65 references that are in the map.
- First posted on arXiv 25 Jan 2025.

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors](https://mlsanity.com/paper/2305.18274/) (2023)
- [Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding](https://mlsanity.com/paper/2211.06956/) (2022)
- [MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data](https://mlsanity.com/paper/2403.11207/) (2024)
- [Natural scene reconstruction from fMRI signals using generative latent diffusion](https://mlsanity.com/paper/2303.05334/) (2023)
- [Scaling laws for language encoding models in fMRI](https://mlsanity.com/paper/2305.11863/) (2023)
- [Brain decoding: toward real-time reconstruction of visual perception](https://mlsanity.com/paper/2310.19812/) (2023)
- [BOLD5000: A public fMRI dataset of 5000 images](https://mlsanity.com/paper/1809.01281/) (2018)
- [Reconstruction of Perceived Images from fMRI Patterns and Semantic Brain Exploration using Instance-Conditioned GANs](https://mlsanity.com/paper/2202.12692/) (2022)
- [Reconstructing Faces from fMRI Patterns using Deep Generative Neural Networks](https://mlsanity.com/paper/1810.03856/) (2018)
- [UniBrain: Unify Image Reconstruction and Captioning All in One Diffusion Model from Human Brain Activity](https://mlsanity.com/paper/2308.07428/) (2023)
- [Self-Supervised Learning of Brain Dynamics from Broad Neuroimaging Data](https://mlsanity.com/paper/2206.11417/) (2022)
- [Controllable Mind Visual Diffusion Model](https://mlsanity.com/paper/2305.10135/) (2023)
- [DCNN-GAN: Reconstructing Realistic Image from fMRI](https://mlsanity.com/paper/1901.07368/) (2019)
- [Neuro-Vision to Language: Enhancing Brain Recording-based Visual Reconstruction and Language Interaction](https://mlsanity.com/paper/2404.19438/) (2024)
- [Spurious reconstruction from brain activity](https://mlsanity.com/paper/2405.10078/) (2024)
- [Semantic Brain Decoding: from fMRI to conceptually similar image reconstruction of visual stimuli](https://mlsanity.com/paper/2212.06726/) (2022)
- [NeuroQuery: comprehensive meta-analysis of human brain mapping](https://mlsanity.com/paper/2002.09261/) (2020)
- [fMRI predictors based on language models of increasing complexity recover brain left lateralization](https://mlsanity.com/paper/2405.17992/) (2024)
- [Deep Recurrent Encoder: A scalable end-to-end network to model brain signals](https://mlsanity.com/paper/2103.02339/) (2021)
- [Inferring brain-computational mechanisms with models of activity measurements](https://mlsanity.com/paper/1608.02027/) (2016)

## Built on by (most cited first, 22 of 22)

- [NeuralBench: A Unifying Framework to Benchmark NeuroAI Models](https://mlsanity.com/paper/2605.08495/) (2026)
- [Disentangling the Factors of Convergence between Brains and Computer Vision Models](https://mlsanity.com/paper/2508.18226/) (2025)
- [A foundation model of vision, audition, and language for in-silico neuroscience](https://mlsanity.com/paper/2605.04326/) (2026)
- [EEG Foundation Challenge: From Cross-Task to Cross-Subject EEG Decoding](https://mlsanity.com/paper/2506.19141/) (2025)
- [Is Limited Participant Diversity Impeding EEG-based Machine Learning?](https://mlsanity.com/paper/2503.13497/) (2025)
- [DIVER-1: Scaling Intracranial EEG Foundation Models for Transferable Representations](https://mlsanity.com/paper/2512.19097/) (2025)
- [Foundation Models for Cross-Domain EEG Analysis Application: A Survey](https://mlsanity.com/paper/2508.15716/) (2025)
- [Mitigating Subject Dependency in EEG Decoding with Subject-Specific Low-Rank Adapters](https://mlsanity.com/paper/2510.08059/) (2025)
- [MEGConformer: Conformer-Based MEG Decoder for Robust Speech and Phoneme Classification](https://mlsanity.com/paper/2512.01443/) (2025)
- [Scaling and context steer LLMs along the same computational path as the human brain](https://mlsanity.com/paper/2512.01591/) (2025)
- [ENIGMA: EEG-to-Image in 15 Minutes Using Less Than 1% of the Parameters](https://mlsanity.com/paper/2602.10361/) (2026)
- [Boosting Brain-to-Image Decoding with TRIBE v2 Data Augmentation](https://mlsanity.com/paper/2606.06345/) (2026)
- [Learning aligned EEG representations with subject-specific encoders](https://mlsanity.com/paper/2606.16462/) (2026)
- [Simple Models, Rich Representations: Visual Decoding from Primate Intracortical Neural Signals](https://mlsanity.com/paper/2601.11108/) (2026)
- [MEG-to-MEG Transfer Learning and Cross-Task Speech/Silence Detection with Limited Data](https://mlsanity.com/paper/2602.18253/) (2026)
- [Toward a Multi-View Brain Network Foundation Model: Cross-View Consistency Learning Across Arbitrary Atlases](https://mlsanity.com/paper/2603.20348/) (2026)
- [How Much is Brain Data Worth for Machine Learning?](https://mlsanity.com/paper/2605.09243/) (2026)
- [DANCE: Detect and Classify Events in EEG](https://mlsanity.com/paper/2605.10688/) (2026)
- [Visualizing the Invisible: Generative Visual Grounding Empowers Universal EEG Understanding in MLLMs](https://mlsanity.com/paper/2605.18172/) (2026)
- [Real-time Reconstruction of Human Visual Perception from fMRI](https://mlsanity.com/paper/2607.22753/) (2026)
- [Beyond Trial Averaging: Anchoring Neural and Visual Representations for Few-Repetition Brain-to-Image Retrieval](https://mlsanity.com/paper/2608.19128/) (2026)
- [Training and Scaling Compute-Optimal Physiological Waveform Foundation Models](https://mlsanity.com/paper/2610.05649/) (2026)

## Abstract (arXiv)

Generative AI has recently propelled the decoding of images from brain activity. How do these approaches scale with the amount and type of neural recordings? Here, we systematically compare image decoding from four types of non-invasive devices: electroencephalography (EEG), magnetoencephalography (MEG), high-field functional Magnetic Resonance Imaging (3T fMRI) and ultra-high field (7T) fMRI. For this, we evaluate decoding models on the largest benchmark to date, encompassing 8 public datasets, 84 volunteers, 498 hours of brain recording and 2.3 million brain responses to natural images. Unlike previous work, we focus on single-trial decoding performance to simulate real-time settings. This systematic comparison reveals three main findings. First, the most precise neuroimaging devices tend to yield the best decoding performances, when the size of the training sets are similar. However, the gain enabled by deep learning - in comparison to linear models - is obtained with the noisiest devices. Second, we do not observe any plateau of decoding performance as the amount of training data increases. Rather, decoding performance scales log-linearly with the amount of brain recording. Third, this scaling law primarily depends on the amount of data per subject. However, little decoding gain is observed by increasing the number of subjects. Overall, these findings delineate the path most suitable to scale the decoding of images from non-invasive brain recordings.