# BioSerenity-E1: a self-supervised EEG model for medical applications

arXiv: https://arxiv.org/abs/2503.10362 · HTML: https://mlsanity.com/paper/2503.10362/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 91 references that are in the map.
- First posted on arXiv 13 Mar 2025.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI](https://mlsanity.com/paper/2405.18765/) (2024)
- [BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://mlsanity.com/paper/2101.12037/) (2021)
- [CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding](https://mlsanity.com/paper/2412.07236/) (2024)
- [Deep learning-based electroencephalography analysis: a systematic review](https://mlsanity.com/paper/1901.05498/) (2019)
- [Transformer-based Spatial-Temporal Feature Learning for EEG Decoding](https://mlsanity.com/paper/2106.11170/) (2021)
- [NeuroLM: A Universal Multi-task Foundation Model for Bridging the Gap between Language and EEG Signals](https://mlsanity.com/paper/2409.00101/) (2024)
- [The Temple University Hospital Seizure Detection Corpus](https://mlsanity.com/paper/1801.08085/) (2018)
- [Self-Supervised Graph Neural Networks for Improved Electroencephalographic Seizure Analysis](https://mlsanity.com/paper/2104.08336/) (2021)
- [Neuro-GPT: Towards A Foundation Model for EEG](https://mlsanity.com/paper/2311.03764/) (2023)
- [BrainBERT: Self-supervised representation learning for intracranial recordings](https://mlsanity.com/paper/2302.14367/) (2023)
- [MAEEG: Masked Auto-encoder for EEG Representation Learning](https://mlsanity.com/paper/2211.02625/) (2022)
- [Theory and Experiments on Vector Quantized Autoencoders](https://mlsanity.com/paper/1805.11063/) (2018)
- [Transformer Convolutional Neural Networks for Automated Artifact Detection in Scalp EEG](https://mlsanity.com/paper/2208.02405/) (2022)
- [EEGFormer: Towards Transferable and Interpretable Large-Scale EEG Foundation Model](https://mlsanity.com/paper/2401.10278/) (2024)
- [EEG2Rep: Enhancing Self-supervised EEG Representation Through Informative Masked Inputs](https://mlsanity.com/paper/2402.17772/) (2024)
- [Improved EEG Event Classification Using Differential Energy](https://mlsanity.com/paper/1801.02477/) (2018)
- [Modeling Multivariate Biosignals With Graph Neural Networks and Structured State Space Models](https://mlsanity.com/paper/2211.11176/) (2022)
- [MBrain: A Multi-channel Self-Supervised Learning Framework for Brain Signals](https://mlsanity.com/paper/2306.13102/) (2023)
- [BrainWave: A Brain Signal Foundation Model for Clinical Applications](https://mlsanity.com/paper/2402.10251/) (2024)
- [Temporal Graph Convolutional Networks for Automatic Seizure Detection](https://mlsanity.com/paper/1905.01375/) (2019)
- [BIOT: Cross-data Biosignal Learning in the Wild](https://mlsanity.com/paper/2305.10351/) (2023)
- [EdVAE: Mitigating Codebook Collapse with Evidential Discrete Variational Autoencoders](https://mlsanity.com/paper/2310.05718/) (2023)

## Built on by (most cited first, 4 of 4)

- [Automatic detection of abnormal clinical EEG: comparison of a finetuned foundation model with two deep learning models](https://mlsanity.com/paper/2505.21507/) (2025)
- [Epileptic Seizure Prediction Using Patient-Adaptive Transformer Networks](https://mlsanity.com/paper/2603.26821/) (2026)
- [DiffEEG: A Self-Supervised Denoising Diffusion Model for Learning EEG Generic Representations](https://mlsanity.com/paper/2607.11578/) (2026)
- [SpecBraM: What Should an EEG Foundation Model Predict? Masked Band-Power Prediction versus Waveform Reconstruction](https://mlsanity.com/paper/2610.07484/) (2026)

## Abstract (arXiv)

Electroencephalography (EEG) serves as an essential diagnostic tool in neurology; however, its accurate manual interpretation is a time-intensive process that demands highly specialized expertise, which remains relatively scarce and not consistently accessible. To address these limitations, the implementation of automated pre-screening and analysis systems for EEG data holds considerable promise. Advances in self-supervised learning made it possible to pre-train complex deep learning architectures on large volumes of unlabeled EEG data to learn generalizable representations, that can later be used to enhance performance on multiple tasks while needing less downstream data. In the present paper, we introduce BioSerenity-E1, the first of a family of self-supervised foundation models for clinical EEG applications that combines spectral tokenization with masked prediction to achieve state-of-the-art performance across relevant diagnostic tasks. The two-phase self-supervised pretraining framework initially acquires compressed EEG representations via a transformer-based VQ-VAE architecture designed to reconstruct log-multitaper spectral projections, then implements extensive (70% block) masked token prediction to force the model to learn complex spatiotemporal dependencies in EEG signals. BioSerenity-E1 achieves strong performance across three clinical tasks, either in line or above state-of-the-art methods: seizure detection (AUROC = 0.926, Sensitivity = 0.909), normal/abnormal classification (AUPRC = 0.970 on proprietary data; 0.910 on TUH-Abnormal), and multiclass pathology differentiation on unbalanced data (Weighted F1 = 0.730). The utility of BioSerenity-E1 is further confirmed in low-data regimes scenarios, showing clear improvements in AUPRC (from +2% to 17%) when trained on less than 10% of the available data.