# Contrastive Learning of Subject-Invariant EEG Representations for Cross-Subject Emotion Recognition

arXiv: https://arxiv.org/abs/2109.09559 · HTML: https://mlsanity.com/paper/2109.09559/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 120 references that are in the map.
- First posted on arXiv 20 Sep 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Uncovering the structure of clinical EEG signals with self-supervised learning](https://mlsanity.com/paper/2007.16104/) (2020)
- [EEG-Based Emotion Recognition Using Regularized Graph Neural Networks](https://mlsanity.com/paper/1907.07835/) (2019)
- [Transfer Learning for EEG-Based Brain-Computer Interfaces: A Review of Progress Made Since 2016](https://mlsanity.com/paper/2004.06286/) (2020)
- [Identifying Stable Patterns over Time for Emotion Recognition from EEG](https://mlsanity.com/paper/1601.02197/) (2016)
- [A Novel Bi-hemispheric Discrepancy Model for EEG Emotion Recognition](https://mlsanity.com/paper/1906.01704/) (2019)
- [Transfer Learning in Brain-Computer Interfaces](https://mlsanity.com/paper/1512.00296/) (2015)
- [Utilizing Deep Learning Towards Multi-modal Bio-sensing and Vision-based Affective Computing](https://mlsanity.com/paper/1905.07039/) (2019)
- [Pre-training of Graph Neural Network for Modeling Effects of Mutations on Protein-Protein Binding Affinity](https://mlsanity.com/paper/2008.12473/) (2020)
- [Correlated Components Analysis - Extracting Reliable Dimensions in Multivariate Data](https://mlsanity.com/paper/1801.08881/) (2018)
- [Dynamic Feature Scaling for Online Learning of Binary Classifiers](https://mlsanity.com/paper/1407.7584/) (2014)

## Built on by (most cited first, 30 of 60)

- [EEG2Rep: Enhancing Self-supervised EEG Representation Through Informative Masked Inputs](https://mlsanity.com/paper/2402.17772/) (2024)
- [EmT: A Novel Transformer for Generalized Cross-subject EEG Emotion Recognition](https://mlsanity.com/paper/2406.18345/) (2024)
- [Self-supervised Learning for Electroencephalogram: A Systematic Survey](https://mlsanity.com/paper/2401.05446/) (2024)
- [Affective Game Computing: A Survey](https://mlsanity.com/paper/2309.14104/) (2023)
- [Semi-Supervised Dual-Stream Self-Attentive Adversarial Graph Contrastive Learning for Cross-Subject EEG-based Emotion Recognition](https://mlsanity.com/paper/2308.11635/) (2023)
- [Supervised Contrastive Learning for Affect Modelling](https://mlsanity.com/paper/2208.12238/) (2022)
- [LibEER: A Comprehensive Benchmark and Algorithm Library for EEG-based Emotion Recognition](https://mlsanity.com/paper/2410.09767/) (2024)
- [Self-supervised Group Meiosis Contrastive Learning for EEG-Based Emotion Recognition](https://mlsanity.com/paper/2208.00877/) (2022)
- [CSLP-AE: A Contrastive Split-Latent Permutation Autoencoder Framework for Zero-Shot Electroencephalography Signal Conversion](https://mlsanity.com/paper/2311.07788/) (2023)
- [Semantic-aware Contrastive Learning for Electroencephalography-to-Text Generation with Curriculum Learning](https://mlsanity.com/paper/2301.09237/) (2023)
- [Joint Contrastive Learning with Feature Alignment for Cross-Corpus EEG-based Emotion Recognition](https://mlsanity.com/paper/2404.09559/) (2024)
- [Domain Adaptation and Generalization on Functional Medical Images: A Systematic Survey](https://mlsanity.com/paper/2212.03176/) (2022)
- [Temporal Aware Mixed Attention-based Convolution and Transformer Network (MACTN) for EEG Emotion Recognition](https://mlsanity.com/paper/2305.18234/) (2023)
- [A Knowledge-Driven Cross-view Contrastive Learning for EEG Representation](https://mlsanity.com/paper/2310.03747/) (2023)
- [Multi-modal Mood Reader: Pre-trained Model Empowers Cross-Subject Emotion Recognition](https://mlsanity.com/paper/2405.19373/) (2024)
- [Multi-view Multi-label Fine-grained Emotion Decoding from Human Brain Activity](https://mlsanity.com/paper/2211.02629/) (2022)
- [PhysioSync: Temporal and Cross-Modal Contrastive Learning Inspired by Physiological Synchronization for EEG-Based Emotion Recognition](https://mlsanity.com/paper/2504.17163/) (2025)
- [Multi-dataset Joint Pre-training of Emotional EEG Enables Generalizable Affective Computing](https://mlsanity.com/paper/2510.22197/) (2025)
- [Cross-Subject Generalization for EEG Decoding: A Survey of Deep Learning Methods](https://mlsanity.com/paper/2604.27033/) (2026)
- [Efficient Labelling of Affective Video Datasets via Few-Shot & Multi-Task Contrastive Learning](https://mlsanity.com/paper/2308.02173/) (2023)
- [Semi-Supervised End-To-End Contrastive Learning For Time Series Classification](https://mlsanity.com/paper/2310.08848/) (2023)
- [Contrastive Learning of Shared Spatiotemporal EEG Representations Across Individuals for Naturalistic Neuroscience](https://mlsanity.com/paper/2402.14213/) (2024)
- [EEG-SCMM: Soft Contrastive Masked Modeling for Cross-Corpus EEG-Based Emotion Recognition](https://mlsanity.com/paper/2408.09186/) (2024)
- [EEG-ReMinD: Enhancing Neurodegenerative EEG Decoding through Self-Supervised State Reconstruction-Primed Riemannian Dynamics](https://mlsanity.com/paper/2501.08139/) (2025)
- [Evaluation in EEG Emotion Recognition: State-of-the-Art Review and Unified Framework](https://mlsanity.com/paper/2505.18175/) (2025)
- [FreqDGT: Frequency-Adaptive Dynamic Graph Networks with Transformer for Cross-subject EEG Emotion Recognition](https://mlsanity.com/paper/2506.22807/) (2025)
- [EMOD: A Unified EEG Emotion Representation Framework Leveraging V-A Guided Contrastive Learning](https://mlsanity.com/paper/2511.05863/) (2025)
- [Self-supervised Gait-based Emotion Representation Learning from Selective Strongly Augmented Skeleton Sequences](https://mlsanity.com/paper/2405.04900/) (2024)
- [Online Multi-level Contrastive Representation Distillation for Cross-Subject fNIRS Emotion Recognition](https://mlsanity.com/paper/2409.16081/) (2024)
- [Subject Representation Learning from EEG using Graph Convolutional Variational Autoencoders](https://mlsanity.com/paper/2501.16626/) (2025)

## Abstract (arXiv)

EEG signals have been reported to be informative and reliable for emotion recognition in recent years. However, the inter-subject variability of emotion-related EEG signals still poses a great challenge for the practical applications of EEG-based emotion recognition. Inspired by recent neuroscience studies on inter-subject correlation, we proposed a Contrastive Learning method for Inter-Subject Alignment (CLISA) to tackle the cross-subject emotion recognition problem. Contrastive learning was employed to minimize the inter-subject differences by maximizing the similarity in EEG signal representations across subjects when they received the same emotional stimuli in contrast to different ones. Specifically, a convolutional neural network was applied to learn inter-subject aligned spatiotemporal representations from EEG time series in contrastive learning. The aligned representations were subsequently used to extract differential entropy features for emotion classification. CLISA achieved state-of-the-art cross-subject emotion recognition performance on our THU-EP dataset with 80 subjects and the publicly available SEED dataset with 15 subjects. It could generalize to unseen subjects or unseen emotional stimuli in testing. Furthermore, the spatiotemporal representations learned by CLISA could provide insights into the neural mechanisms of human emotion processing.