# Self-supervised EEG Representation Learning for Automatic Sleep Staging

arXiv: https://arxiv.org/abs/2110.15278 · HTML: https://mlsanity.com/paper/2110.15278/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 52 references that are in the map.
- First posted on arXiv 27 Oct 2021.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://mlsanity.com/paper/1902.09229/) (2019)
- [Debiased Contrastive Learning](https://mlsanity.com/paper/2007.00224/) (2020)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [On Mutual Information Maximization for Representation Learning](https://mlsanity.com/paper/1907.13625/) (2019)
- [Learning from Simulated and Unsupervised Images through Adversarial Training](https://mlsanity.com/paper/1612.07828/) (2016)
- [Unsupervised Scalable Representation Learning for Multivariate Time Series](https://mlsanity.com/paper/1901.10738/) (2019)
- [CERT: Contrastive Self-supervised Learning for Language Understanding](https://mlsanity.com/paper/2005.12766/) (2020)
- [Self-supervised Video Representation Learning by Pace Prediction](https://mlsanity.com/paper/2008.05861/) (2020)
- [Uncovering the structure of clinical EEG signals with self-supervised learning](https://mlsanity.com/paper/2007.16104/) (2020)
- [Subject-Aware Contrastive Learning for Biosignals](https://mlsanity.com/paper/2007.04871/) (2020)
- [SLEEPNET: Automated Sleep Staging System via Deep Learning](https://mlsanity.com/paper/1707.08262/) (2017)
- [Does Visual Self-Supervision Improve Learning of Speech Representations for Emotion Recognition?](https://mlsanity.com/paper/2005.01400/) (2020)
- [SLEEPER: interpretable Sleep staging via Prototypes from Expert Rules](https://mlsanity.com/paper/1910.06100/) (2019)
- [ATD: Augmenting CP Tensor Decomposition by Self Supervision](https://mlsanity.com/paper/2106.07900/) (2021)

## Built on by (most cited first, 30 of 32)

- [NeuroLM: A Universal Multi-task Foundation Model for Bridging the Gap between Language and EEG Signals](https://mlsanity.com/paper/2409.00101/) (2024)
- [CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding](https://mlsanity.com/paper/2506.23075/) (2025)
- [Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directions](https://mlsanity.com/paper/2111.08446/) (2021)
- [BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals](https://mlsanity.com/paper/2505.18185/) (2025)
- [LUNA: Efficient and Topology-Agnostic Foundation Model for EEG Signal Analysis](https://mlsanity.com/paper/2510.22257/) (2025)
- [ManyDG: Many-domain Generalization for Healthcare Applications](https://mlsanity.com/paper/2301.08834/) (2023)
- [BIOT: Cross-data Biosignal Learning in the Wild](https://mlsanity.com/paper/2305.10351/) (2023)
- [Self-supervised Learning for Electroencephalogram: A Systematic Survey](https://mlsanity.com/paper/2401.05446/) (2024)
- [FoME: A Foundation Model for EEG using Adaptive Temporal-Lateral Attention Scaling](https://mlsanity.com/paper/2409.12454/) (2024)
- [Neural Codecs as Biosignal Tokenizers](https://mlsanity.com/paper/2510.09095/) (2025)
- [Uni-NTFM: A Unified Foundation Model for EEG Signal Representation Learning](https://mlsanity.com/paper/2509.24222/) (2025)
- [Motor Imagery Classification for Asynchronous EEG-Based Brain-Computer Interfaces](https://mlsanity.com/paper/2412.09006/) (2024)
- [SAMBA: Toward a Long-Context EEG Foundation Model via Spatial Embedding and Differential Mamba](https://mlsanity.com/paper/2511.18571/) (2025)
- [NeuroWeaver: An Autonomous Evolutionary Agent for Exploring the Programmatic Space of EEG Analysis Pipelines](https://mlsanity.com/paper/2602.13473/) (2026)
- [A Survey of Spatio-Temporal EEG data Analysis: from Models to Applications](https://mlsanity.com/paper/2410.08224/) (2024)
- [Deep Learning-Powered Electrical Brain Signals Analysis: Advancing Neurological Diagnostics](https://mlsanity.com/paper/2502.17213/) (2025)
- [Are foundation models useful feature extractors for electroencephalography analysis?](https://mlsanity.com/paper/2502.21086/) (2025)
- [Large EEG-U-Transformer for Time-Step Level Detection Without Pre-Training](https://mlsanity.com/paper/2504.00336/) (2025)
- [Brain-OF: An Omnifunctional Foundation Model for fMRI, EEG and MEG](https://mlsanity.com/paper/2602.23410/) (2026)
- [EEGDM: Learning EEG Representation with Latent Diffusion Model](https://mlsanity.com/paper/2508.20705/) (2025)
- [FEMBA on the Edge: Physiologically-Aware Pre-Training, Quantization, and Deployment of a Bidirectional Mamba EEG Foundation Model on an Ultra-low Power Microcontroller](https://mlsanity.com/paper/2603.26716/) (2026)
- [CaMBRAIN: Real-time, Continuous EEG Inference with Causal State Space Models](https://mlsanity.com/paper/2605.28792/) (2026)
- [Large-scale Foundation Models and Generative AI for BigData Neuroscience](https://mlsanity.com/paper/2310.18377/) (2023)
- [Multimodal Functional Maximum Correlation for Emotion Recognition](https://mlsanity.com/paper/2512.23076/) (2025)
- [One Brain, Omni Modalities: Towards Unified Non-Invasive Brain Decoding with Large Language Models](https://mlsanity.com/paper/2602.21522/) (2026)
- [NeuroNarrator: A Generalist EEG-to-Text Foundation Model for Clinical Interpretation via Spectro-Spatial Grounding and Temporal State-Space Reasoning](https://mlsanity.com/paper/2603.16880/) (2026)
- [B[FM]2: Brain Foundation Model via Flow Matching with SplitUNet](https://mlsanity.com/paper/2606.20812/) (2026)
- [CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding](https://mlsanity.com/paper/2607.09543/) (2026)
- [Interpretable EEG biomarkers with bag-of-waves: Spatial and temporal waveform dictionaries for low-data regimes](https://mlsanity.com/paper/2607.22508/) (2026)
- [BRIDGE-EEG: Bridging Self-Supervised Pretraining and Efficient Deployment for Cross-Dataset EEG Classification](https://mlsanity.com/paper/2609.12218/) (2026)

## Abstract (arXiv)

Background: Deep learning models have shown great success in automating tasks in sleep medicine by learning from carefully annotated Electroencephalogram (EEG) data. However, effectively utilizing a large amount of raw EEG remains a challenge. Objective: In this paper, we aim to learn robust vector representations from massive unlabeled EEG signals, such that the learned vectorized features (1) are expressive enough to replace the raw signals in the sleep staging task; and (2) provide better predictive performance than supervised models in scenarios of fewer labels and noisy samples. Methods: We propose a self-supervised model, named Contrast with the World Representation (ContraWR), for EEG signal representation learning, which uses global statistics from the dataset to distinguish signals associated with different sleep stages. The ContraWR model is evaluated on three real-world EEG datasets that include both at-home and in-lab EEG recording settings. Results: ContraWR outperforms 4 recent self-supervised learning methods on the sleep staging task across 3 large EEG datasets. ContraWR also beats supervised learning when fewer training labels are available (e.g., 4% accuracy improvement when less than 2% data is labeled). Moreover, the model provides informative representative feature structures in 2D projection. Conclusions: We show that ContraWR is robust to noise and can provide high-quality EEG representations for downstream prediction tasks. The proposed model can be generalized to other unsupervised physiological signal learning tasks. Future directions include exploring task-specific data augmentations and combining self-supervised with supervised methods, building upon the initial success of self-supervised learning in this paper.