# NeuralBench: A Unifying Framework to Benchmark NeuroAI Models

arXiv: https://arxiv.org/abs/2605.08495 · HTML: https://mlsanity.com/paper/2605.08495/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 150 references that are in the map.
- First posted on arXiv 8 May 2026.

## Builds on (most cited first, 30 of 42)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI](https://mlsanity.com/paper/2405.18765/) (2024)
- [BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://mlsanity.com/paper/2101.12037/) (2021)
- [CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding](https://mlsanity.com/paper/2412.07236/) (2024)
- [MOABB: Trustworthy algorithm benchmarking for BCIs](https://mlsanity.com/paper/1805.06427/) (2018)
- [REVE: A Foundation Model for EEG -- Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjects](https://mlsanity.com/paper/2510.21585/) (2025)
- [Machine Learning for Neuroimaging with Scikit-Learn](https://mlsanity.com/paper/1412.3919/) (2014)
- [Brain decoding: toward real-time reconstruction of visual perception](https://mlsanity.com/paper/2310.19812/) (2023)
- [Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Masking](https://mlsanity.com/paper/2409.19407/) (2024)
- [A Unified, Scalable Framework for Neural Population Decoding](https://mlsanity.com/paper/2310.16046/) (2023)
- [The Algonauts Project 2023 Challenge: How the Human Brain Makes Sense of Natural Scenes](https://mlsanity.com/paper/2301.03198/) (2023)
- [SwiFT: Swin 4D fMRI Transformer](https://mlsanity.com/paper/2307.05916/) (2023)
- [Machine-Learning-Based Diagnostics of EEG Pathology](https://mlsanity.com/paper/2002.05115/) (2020)
- [Improved EEG Event Classification Using Differential Energy](https://mlsanity.com/paper/1801.02477/) (2018)
- [LUNA: Efficient and Topology-Agnostic Foundation Model for EEG Signal Analysis](https://mlsanity.com/paper/2510.22257/) (2025)
- [2021 BEETL Competition: Advancing Transfer Learning for Subject Independence & Heterogenous EEG Data Sets](https://mlsanity.com/paper/2202.12950/) (2022)
- [EEG-FM-Bench: A Comprehensive Benchmark for the Systematic Evaluation and Diagnostic Analyses of EEG Foundation Models](https://mlsanity.com/paper/2508.17742/) (2025)
- [Scaling laws for decoding images from brain activity](https://mlsanity.com/paper/2501.15322/) (2025)
- [Brain-to-Text Decoding: A Non-invasive Approach via Typing](https://mlsanity.com/paper/2502.17480/) (2025)
- [Brain Foundation Models: A Survey on Advancements in Neural Signal Processing and Brain Discovery](https://mlsanity.com/paper/2503.00580/) (2025)
- [The Algonauts Project 2025 Challenge: How the Human Brain Makes Sense of Multimodal Movies](https://mlsanity.com/paper/2501.00504/) (2024)
- [The Algonauts Project 2021 Challenge: How the Human Brain Makes Sense of a World in Motion](https://mlsanity.com/paper/2104.13714/) (2021)
- [AdaBrain-Bench: Benchmarking Brain Foundation Models for Brain-Computer Interface Applications](https://mlsanity.com/paper/2507.09882/) (2025)
- [A Plug&Play P300 BCI Using Information Geometry](https://mlsanity.com/paper/1409.0107/) (2014)
- [The Sensorium competition on predicting large-scale mouse primary visual cortex activity](https://mlsanity.com/paper/2206.08666/) (2022)

## Built on by (most cited first, 16 of 16)

- [Stacked LoRA for Subject-Adaptive EEG Foundation Models in Motor Imagery Decoding](https://mlsanity.com/paper/2607.03094/) (2026)
- [Understanding and Correcting Low-Frequency Bias in EEG Foundation Model](https://mlsanity.com/paper/2608.01898/) (2026)
- [Lightweight Adaptation of EEG Foundation Models for Stroke Motor Imagery Decoding: Domain Shift and Subject-Level Robustness](https://mlsanity.com/paper/2609.00282/) (2026)
- [NeuroShield: A Device-Agnostic Foundation Model for EEG Authentication](https://mlsanity.com/paper/2606.20673/) (2026)
- [ZUNA1.1: A more flexible EEG foundation model for Denoising and Super-resolution](https://mlsanity.com/paper/2607.27308/) (2026)
- [BrainBench: Benchmarking Large Language Models for Comprehensive EEG Understanding](https://mlsanity.com/paper/2608.04156/) (2026)
- [Continuous-Latent Predictive Modeling with Semantic Alignment for EEG-Language Foundation Models](https://mlsanity.com/paper/2608.11656/) (2026)
- [ReMAP: Self-supervised learning to unveil brain representations and vulnerability](https://mlsanity.com/paper/2608.22042/) (2026)
- [A Roadmap for MEG Foundation Models](https://mlsanity.com/paper/2609.04461/) (2026)
- [Matched-Input Estimates Differ in Sign Across Architectures: Auditing EEG Foundation Models on Motor Imagery](https://mlsanity.com/paper/2609.23924/) (2026)
- [Neural State Prediction: Obstructing Shortcut Learning in EEG Foundation Models](https://mlsanity.com/paper/2609.31167/) (2026)
- [EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis](https://mlsanity.com/paper/2609.31632/) (2026)
- [Benchmarking EEG Foundation Models at Scale: Lessons from 20,000 Evaluations](https://mlsanity.com/paper/2609.32743/) (2026)
- [What masking geometry works best for EEG foundation models?](https://mlsanity.com/paper/2609.33487/) (2026)
- [Removing Timing Shortcuts Improves Non-Invasive Brain-to-Text](https://mlsanity.com/paper/2609.40359/) (2026)
- [GeoNIRS: Geometry-Aware Pretraining for Heterogeneous fNIRS Representation Learning](https://mlsanity.com/paper/2610.17845/) (2026)

## Abstract (arXiv)

Deep learning and large public datasets have recently catalyzed the proliferation of AI models for processing brain recordings. However, systematically evaluating these models remains a challenge: not only do the preprocessing pipelines, training and finetuning approaches largely vary across studies, but their downstream evaluation is often limited to small sets of tasks and/or datasets. Here, we present NeuralBench: a unified framework for benchmarking AI models of brain activity. We accompany this framework with NeuralBench-EEG v1.0 -- a large EEG benchmark that includes 36 electroencephalography (EEG) tasks and 14 deep learning architectures, and is evaluated on 94 datasets accessed through a standardized interface. This first EEG-focused release already highlights two main findings. First, current foundation models only marginally outperform task-specific models. Second, a large set of tasks (e.g. cognitive decoding, clinical predictions) remain highly challenging, even for the best models. Critically, NeuralBench is designed for the integration of new tasks, datasets, models, and neuroimaging modalities, as illustrated by preliminary extensions to MEG and fMRI datasets and models. Through this white paper, we invite the community to expand this open-source framework and work together toward a unified benchmarking standard for neuroimaging models.