# Improving self-supervised pretraining models for epileptic seizure detection from EEG data

arXiv: https://arxiv.org/abs/2207.06911 · HTML: https://mlsanity.com/paper/2207.06911/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 28 Jun 2022.

## Builds on (most cited first, 20 of 20)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://mlsanity.com/paper/1902.09229/) (2019)
- [BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://mlsanity.com/paper/2101.12037/) (2021)
- [Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data](https://mlsanity.com/paper/2010.03622/) (2020)
- [Machine Learning on Graphs: A Model and Comprehensive Taxonomy](https://mlsanity.com/paper/2005.03675/) (2020)
- [Uncovering the structure of clinical EEG signals with self-supervised learning](https://mlsanity.com/paper/2007.16104/) (2020)
- [The Temple University Hospital Seizure Detection Corpus](https://mlsanity.com/paper/1801.08085/) (2018)
- [Self-Supervised Graph Neural Networks for Improved Electroencephalographic Seizure Analysis](https://mlsanity.com/paper/2104.08336/) (2021)
- [A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks](https://mlsanity.com/paper/2010.03648/) (2020)
- [Machine Learning for Predicting Epileptic Seizures Using EEG Signals: A Review](https://mlsanity.com/paper/2002.01925/) (2020)
- [EEG-GCNN: Augmenting Electroencephalogram-based Neurological Disease Diagnosis using a Domain-guided Graph Convolutional Neural Network](https://mlsanity.com/paper/2011.12107/) (2020)
- [Neonatal seizure detection from raw multi-channel EEG using a fully convolutional architecture](https://mlsanity.com/paper/2105.13854/) (2021)
- [Representation Learning for Sequence Data with Deep Autoencoding Predictive Components](https://mlsanity.com/paper/2010.03135/) (2020)

## Built on by (most cited first, 3 of 3)

- [Self-supervised Learning for Electroencephalogram: A Systematic Survey](https://mlsanity.com/paper/2401.05446/) (2024)
- [Region-Aware Reconstruction Strategy for Pre-training fMRI Foundation Model](https://mlsanity.com/paper/2511.00443/) (2025)
- [Using Explainable AI for EEG-based Reduced Montage Neonatal Seizure Detection](https://mlsanity.com/paper/2406.16908/) (2024)

## Abstract (arXiv)

There is abundant medical data on the internet, most of which are unlabeled. Traditional supervised learning algorithms are often limited by the amount of labeled data, especially in the medical domain, where labeling is costly in terms of human processing and specialized experts needed to label them. They are also prone to human error and biased as a select few expert annotators label them. These issues are mitigated by Self-supervision, where we generate pseudo-labels from unlabelled data by seeing the data itself. This paper presents various self-supervision strategies to enhance the performance of a time-series based Diffusion convolution recurrent neural network (DCRNN) model. The learned weights in the self-supervision pretraining phase can be transferred to the supervised training phase to boost the model's prediction capability. Our techniques are tested on an extension of a Diffusion Convolutional Recurrent Neural network (DCRNN) model, an RNN with graph diffusion convolutions, which models the spatiotemporal dependencies present in EEG signals. When the learned weights from the pretraining stage are transferred to a DCRNN model to determine whether an EEG time window has a characteristic seizure signal associated with it, our method yields an AUROC score 1.56% than the current state-of-the-art models on the TUH EEG seizure corpus.