# Bridging the Gap Between Patient-specific and Patient-independent Seizure Prediction via Knowledge Distillation

arXiv: https://arxiv.org/abs/2202.12598 · HTML: https://mlsanity.com/paper/2202.12598/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation.
- Builds on 22 of its 65 references that are in the map.
- First posted on arXiv 25 Feb 2022.

## Placed in

- Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/)

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons](https://mlsanity.com/paper/1811.03233/) (2018)
- [Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning](https://mlsanity.com/paper/1702.08811/) (2017)
- [Learning Lightweight Lane Detection CNNs by Self Attention Distillation](https://mlsanity.com/paper/1908.00821/) (2019)
- [Deep learning-based electroencephalography analysis: a systematic review](https://mlsanity.com/paper/1901.05498/) (2019)
- [Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1705.00609/) (2017)
- [Snapshot Distillation: Teacher-Student Optimization in One Generation](https://mlsanity.com/paper/1812.00123/) (2018)
- [Understanding and Improving Knowledge Distillation](https://mlsanity.com/paper/2002.03532/) (2020)
- [Self-supervised Knowledge Distillation Using Singular Value Decomposition](https://mlsanity.com/paper/1807.06819/) (2018)
- [Conditional Teacher-Student Learning](https://mlsanity.com/paper/1904.12399/) (2019)
- [Adversarial Representation Learning for Robust Patient-Independent Epileptic Seizure Detection](https://mlsanity.com/paper/1909.10868/) (2019)
- [An End-to-End Deep Learning Approach for Epileptic Seizure Prediction](https://mlsanity.com/paper/2108.07453/) (2021)
- [Unsupervised Domain Adaptation for Cross-Subject Few-Shot Neurological Symptom Detection](https://mlsanity.com/paper/2103.00606/) (2021)
- [Deep Belief Networks used on High Resolution Multichannel Electroencephalography Data for Seizure Detection](https://mlsanity.com/paper/1708.08430/) (2017)
- [C2SP-Net: Joint Compression and Classification Network for Epilepsy Seizure Prediction](https://mlsanity.com/paper/2110.13674/) (2021)

## Built on by (most cited first, 3 of 3)

- [Neuro-BERT: Rethinking Masked Autoencoding for Self-supervised Neurological Pretraining](https://mlsanity.com/paper/2204.12440/) (2022)
- [Transfer Learning on Electromyography (EMG) Tasks: Approaches and Beyond](https://mlsanity.com/paper/2210.06295/) (2022)
- [Federated Few-Shot Learning for Epileptic Seizure Detection Under Privacy Constraints](https://mlsanity.com/paper/2512.13717/) (2025)

## Abstract (arXiv)

Objective. Deep neural networks (DNNs) have shown unprecedented success in various brain-machine interface applications such as epileptic seizure prediction. However, existing approaches typically train models in a patient-specific fashion due to the highly personalized characteristics of epileptic signals. Therefore, only a limited number of labeled recordings from each subject can be used for training. As a consequence, current DNN based methods demonstrate poor generalization ability to some extent due to the insufficiency of training data. On the other hand, patient-independent models attempt to utilize more patient data to train a universal model for all patients by pooling patient data together. Despite different techniques applied, results show that patient-independent models perform worse than patient-specific models due to high individual variation across patients. A substantial gap thus exists between patient-specific and patient-independent models. Approach. In this paper, we propose a novel training scheme based on knowledge distillation which makes use of a large amount of data from multiple subjects. It first distills informative features from signals of all available subjects with a pre-trained general model. A patient-specific model can then be obtained with the help of distilled knowledge and additional personalized data. Main results. Four state-of-the-art seizure prediction methods are trained on the Children's Hospital of Boston-MIT sEEG database with our proposed scheme. The resulting accuracy, sensitivity, and false prediction rate show that our proposed training scheme consistently improves the prediction performance of state-of-the-art methods by a large margin. Significance. The proposed training scheme significantly improves the performance of patient-specific seizure predictors and bridges the gap between patient-specific and patient-independent predictors.