# BrainGPT: Unleashing the Potential of EEG Generalist Foundation Model by Autoregressive Pre-training

arXiv: https://arxiv.org/abs/2410.19779 · HTML: https://mlsanity.com/paper/2410.19779/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 49 references that are in the map.
- First posted on arXiv 14 Oct 2024.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Language Modeling with Gated Convolutional Networks](https://mlsanity.com/paper/1612.08083/) (2016)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI](https://mlsanity.com/paper/2405.18765/) (2024)
- [TSception: Capturing Temporal Dynamics and Spatial Asymmetry from EEG for Emotion Recognition](https://mlsanity.com/paper/2104.02935/) (2021)
- [Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks](https://mlsanity.com/paper/1511.06448/) (2015)
- [LGGNet: Learning from Local-Global-Graph Representations for Brain-Computer Interface](https://mlsanity.com/paper/2105.02786/) (2021)
- [Learning from Heterogeneous EEG Signals with Differentiable Channel Reordering](https://mlsanity.com/paper/2010.13694/) (2020)

## Built on by (most cited first, 6 of 6)

- [EEG-FM-Compass: Progress, Benchmarking, and Future Directions for EEG Foundation Models](https://mlsanity.com/paper/2601.17883/) (2026)
- [Large-Scale AI and Foundation Models for Neuroscience: A Comprehensive Review](https://mlsanity.com/paper/2510.16658/) (2025)
- [MyoSem: Aligning Electromyography with Natural-Language Action Semantics](https://mlsanity.com/paper/2606.00174/) (2026)
- [Behavioral Latency as Weak Event-Time Supervision for EEG Reaction-Time Decoding](https://mlsanity.com/paper/2608.29428/) (2026)
- [Dataset-Dependent Effects of Cross-Depth Aggregation and Soft-Routed Experts in EEG Foundation Model Fine-Tuning](https://mlsanity.com/paper/2609.17886/) (2026)
- [EMG-GPT: Predictive Pretraining on Residual-Quantized EMG Tokens for Hand Pose Estimation](https://mlsanity.com/paper/2610.05235/) (2026)

## Abstract (arXiv)

Electroencephalogram (EEG) signals are pivotal in providing insights into spontaneous brain activity, highlighting their significant importance in neuroscience research. However, the exploration of versatile EEG models is constrained by diverse data formats, outdated pre-training paradigms, and limited transfer learning methods, only leading to specialist models on single dataset. In this paper, we introduce EEGPT, the first generalist EEG foundation model designed to address these challenges. First, we propose an electrode-wise modeling strategy that treats each electrode as a fundamental unit, enabling the integration of diverse EEG datasets collected from up to 138 electrodes, amassing 37.5M pre-training samples. Second, we develop the first autoregressive EEG pre-trained model, moving away from traditional masked autoencoder approaches to a next signal prediction task that better captures the sequential and temporal dependencies of EEG data. We also explore scaling laws with model up to 1.1B parameters: the largest in EEG research to date. Third, we introduce a multi-task transfer learning paradigm using a learnable electrode graph network shared across tasks, which for the first time confirms multi-task compatibility and synergy. As the first generalist EEG foundation model, EEGPT shows broad compatibility with various signal acquisition devices, subjects, and tasks. It supports up to 138 electrodes and any combination thereof as input. Furthermore, we simultaneously evaluate it on 5 distinct tasks across 12 benchmarks. EEGPT consistently outperforms existing specialist models across all downstream tasks, with its effectiveness further validated through extensive ablation studies. This work sets a new direction for generalist EEG modeling, offering improved scalability, transferability, and adaptability for a wide range of EEG applications. The code and models will be released.