# EEG-FM-Bench: A Comprehensive Benchmark for the Systematic Evaluation and Diagnostic Analyses of EEG Foundation Models

arXiv: https://arxiv.org/abs/2508.17742 · HTML: https://mlsanity.com/paper/2508.17742/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 60 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI](https://mlsanity.com/paper/2405.18765/) (2024)
- [BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://mlsanity.com/paper/2101.12037/) (2021)
- [CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding](https://mlsanity.com/paper/2412.07236/) (2024)
- [BrainBERT: Self-supervised representation learning for intracranial recordings](https://mlsanity.com/paper/2302.14367/) (2023)
- [CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding](https://mlsanity.com/paper/2506.23075/) (2025)
- [EEG2Rep: Enhancing Self-supervised EEG Representation Through Informative Masked Inputs](https://mlsanity.com/paper/2402.17772/) (2024)
- [Inter-database validation of a deep learning approach for automatic sleep scoring](https://mlsanity.com/paper/2009.10365/) (2020)
- [EEGdenoiseNet: A benchmark dataset for end-to-end deep learning solutions of EEG denoising](https://mlsanity.com/paper/2009.11662/) (2020)
- [Improved EEG Event Classification Using Differential Energy](https://mlsanity.com/paper/1801.02477/) (2018)
- [Mantis: Lightweight Foundation Model for Time Series Classification](https://mlsanity.com/paper/2502.15637/) (2025)
- [2021 BEETL Competition: Advancing Transfer Learning for Subject Independence & Heterogenous EEG Data Sets](https://mlsanity.com/paper/2202.12950/) (2022)
- [AdaBrain-Bench: Benchmarking Brain Foundation Models for Brain-Computer Interface Applications](https://mlsanity.com/paper/2507.09882/) (2025)
- [Are Large Brainwave Foundation Models Capable Yet? Insights from Fine-tuning](https://mlsanity.com/paper/2507.01196/) (2025)
- [EEG Foundation Challenge: From Cross-Task to Cross-Subject EEG Decoding](https://mlsanity.com/paper/2506.19141/) (2025)
- [A Simple Review of EEG Foundation Models: Datasets, Advancements and Future Perspectives](https://mlsanity.com/paper/2504.20069/) (2025)
- [LibEER: A Comprehensive Benchmark and Algorithm Library for EEG-based Emotion Recognition](https://mlsanity.com/paper/2410.09767/) (2024)
- [GNN4EEG: A Benchmark and Toolkit for Electroencephalography Classification with Graph Neural Network](https://mlsanity.com/paper/2309.15515/) (2023)
- [Benchmarking of EEG Analysis Techniques for Parkinson's Disease Diagnosis: A Comparison between Traditional ML Methods and Foundation DL Methods](https://mlsanity.com/paper/2507.13716/) (2025)

## Built on by (most cited first, 23 of 23)

- [NeuralBench: A Unifying Framework to Benchmark NeuroAI Models](https://mlsanity.com/paper/2605.08495/) (2026)
- [Uni-NTFM: A Unified Foundation Model for EEG Signal Representation Learning](https://mlsanity.com/paper/2509.24222/) (2025)
- [DeeperBrain: A Neuro-Grounded EEG Foundation Model Towards Universal BCI](https://mlsanity.com/paper/2601.06134/) (2026)
- [Benchmarking ECG FMs: A Reality Check Across Clinical Tasks](https://mlsanity.com/paper/2509.25095/) (2025)
- [NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces](https://mlsanity.com/paper/2605.14698/) (2026)
- [Laya: A LeJEPA Approach to EEG via Latent Prediction over Reconstruction](https://mlsanity.com/paper/2603.16281/) (2026)
- [EEG-FM-Audit: A Systematic Evaluation and Analysis Pipeline for EEG Foundation Models](https://mlsanity.com/paper/2605.26910/) (2026)
- [The Identity Trap in EEG Foundation Models: A Diagnostic Audit](https://mlsanity.com/paper/2606.06647/) (2026)
- [SCOPE: Structured Prototype-Guided Adaptation for EEG Foundation Models with Limited Labels](https://mlsanity.com/paper/2602.17251/) (2026)
- [SpecMoE: Spectral Mixture-of-Experts Foundation Model for Cross-Species EEG Decoding](https://mlsanity.com/paper/2603.16739/) (2026)
- [NS-Copilot: An LLM-Driven Agent System for Autonomous Neuroscience Analysis](https://mlsanity.com/paper/2609.01971/) (2026)
- [MANAS-2: Constrained Reconstruction for EEG Foundation Models](https://mlsanity.com/paper/2609.13717/) (2026)
- [PRISM: Exploring Heterogeneous Pretrained EEG Foundation Model Transfer to Clinical Differential Diagnosis](https://mlsanity.com/paper/2603.02268/) (2026)
- [CAMEL-CLIP: Channel-aware Multimodal Electroencephalography-text Alignment for Generalizable Brain Foundation Models](https://mlsanity.com/paper/2603.13272/) (2026)
- [EvoBrain: Continual Learning of EEG Foundation Models Across Heterogeneous BCI Tasks](https://mlsanity.com/paper/2606.01767/) (2026)
- [ZUNA1.1: A more flexible EEG foundation model for Denoising and Super-resolution](https://mlsanity.com/paper/2607.27308/) (2026)
- [EEG-AS: Instance-Level Foundation Model Selection for EEG Foundation Models via Behavior Reconstruction](https://mlsanity.com/paper/2609.00653/) (2026)
- [A Roadmap for MEG Foundation Models](https://mlsanity.com/paper/2609.04461/) (2026)
- [Matched-Input Estimates Differ in Sign Across Architectures: Auditing EEG Foundation Models on Motor Imagery](https://mlsanity.com/paper/2609.23924/) (2026)
- [Benchmarking EEG Foundation Models at Scale: Lessons from 20,000 Evaluations](https://mlsanity.com/paper/2609.32743/) (2026)
- [PhysioTRACE: Provenance-Aware Stress Tests for Physiological Foundation Models](https://mlsanity.com/paper/2609.34466/) (2026)
- [Representational and Functional Robustness to Electrode Montages in EEG Foundation Models](https://mlsanity.com/paper/2609.36288/) (2026)
- [PPG-Bench: A Unified Benchmark for PPG Foundation Models](https://mlsanity.com/paper/2610.13435/) (2026)

## Abstract (arXiv)

Electroencephalography foundation models (EEG-FMs) have advanced brain signal analysis, but the lack of standardized evaluation benchmarks impedes model comparison and scientific progress. Current evaluations rely on inconsistent protocols that render cross-model comparisons unreliable, while a lack of diagnostic analyses obscures the internal mechanisms driving transfer efficiency and scaling behaviors. To address this, we introduce EEG-FM-Bench, a unified system for the standardized evaluation of EEG-FMs. The benchmark integrates 14 datasets across 10 paradigms and incorporates diverse experimental settings, including multiple fine-tuning strategies, task organizations, and classifier configurations, supported by tools for gradient and representation analysis. Our experiments and analysis reveal several critical insights: (1) multi-task learning often acts as a useful regularizer that mitigates overfitting in data-scarce EEG contexts, although negative transfer can arise under specific task paradigms; (2) pre-training efficiency is currently limited by gradient conflicts between reconstruction objectives and downstream tasks; (3) under released checkpoints and a matched downstream protocol, model or data scale alone does not fully explain transfer performance, while objective alignment, adaptation compatibility, and EEG-specific design appear to be important factors. This benchmark enables fair comparison and reproducible analysis, providing a step toward fairer comparison and more interpretable analysis of EEG-FMs. Code is available at https://github.com/xw1216/EEG-FM-Bench.