# Self-supervised ECG Representation Learning for Emotion Recognition

arXiv: https://arxiv.org/abs/2002.03898 · HTML: https://mlsanity.com/paper/2002.03898/

## Key facts

- Cited by 99 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2020.

## Built on by (most cited first, 30 of 99)

- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [Large-scale Training of Foundation Models for Wearable Biosignals](https://mlsanity.com/paper/2312.05409/) (2023)
- [Guiding Masked Representation Learning to Capture Spatio-Temporal Relationship of Electrocardiogram](https://mlsanity.com/paper/2402.09450/) (2024)
- [Self-supervised representation learning from 12-lead ECG data](https://mlsanity.com/paper/2103.12676/) (2021)
- [PaPaGei: Open Foundation Models for Optical Physiological Signals](https://mlsanity.com/paper/2410.20542/) (2024)
- [Deep Gait Recognition: A Survey](https://mlsanity.com/paper/2102.09546/) (2021)
- [TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoders](https://mlsanity.com/paper/2303.00320/) (2023)
- [Self-supervised Contrastive Representation Learning for Semi-supervised Time-Series Classification](https://mlsanity.com/paper/2208.06616/) (2022)
- [3KG: Contrastive Learning of 12-Lead Electrocardiograms using Physiologically-Inspired Augmentations](https://mlsanity.com/paper/2106.04452/) (2021)
- [Intra-Inter Subject Self-supervised Learning for Multivariate Cardiac Signals](https://mlsanity.com/paper/2109.08908/) (2021)
- [SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data](https://mlsanity.com/paper/2102.06073/) (2021)
- [MHCCL: Masked Hierarchical Cluster-Wise Contrastive Learning for Multivariate Time Series](https://mlsanity.com/paper/2212.01141/) (2022)
- [A Transformer Architecture for Stress Detection from ECG](https://mlsanity.com/paper/2108.09737/) (2021)
- [ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Text](https://mlsanity.com/paper/2405.19366/) (2024)
- [Self-supervised Contrastive Learning of Multi-view Facial Expressions](https://mlsanity.com/paper/2108.06723/) (2021)
- [ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition](https://mlsanity.com/paper/2202.00758/) (2022)
- [COCOA: Cross Modality Contrastive Learning for Sensor Data](https://mlsanity.com/paper/2208.00467/) (2022)
- [FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification](https://mlsanity.com/paper/2302.09818/) (2023)
- [Region-Disentangled Diffusion Model for High-Fidelity PPG-to-ECG Translation](https://mlsanity.com/paper/2308.13568/) (2023)
- [LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speech](https://mlsanity.com/paper/2309.05472/) (2023)
- [Transformer-based Self-supervised Multimodal Representation Learning for Wearable Emotion Recognition](https://mlsanity.com/paper/2303.17611/) (2023)
- [Label-efficient Time Series Representation Learning: A Review](https://mlsanity.com/paper/2302.06433/) (2023)
- [Refashioning Emotion Recognition Modelling: The Advent of Generalised Large Models](https://mlsanity.com/paper/2308.11578/) (2023)
- [Wearable Accelerometer Foundation Models for Health via Knowledge Distillation](https://mlsanity.com/paper/2412.11276/) (2024)
- [Towards quantitative precision for ECG analysis: Leveraging state space models, self-supervision and patient metadata](https://mlsanity.com/paper/2308.15291/) (2023)
- [Self-supervised Learning for Label-Efficient Sleep Stage Classification: A Comprehensive Evaluation](https://mlsanity.com/paper/2210.06286/) (2022)
- [Towards Domain Generalization for ECG and EEG Classification: Algorithms and Benchmarks](https://mlsanity.com/paper/2303.11338/) (2023)
- [Unsupervised Representation Learning for Time Series: A Review](https://mlsanity.com/paper/2308.01578/) (2023)
- [Unlocking the diagnostic potential of electrocardiograms through information transfer from cardiac magnetic resonance imaging](https://mlsanity.com/paper/2308.05764/) (2023)
- [Multimodal Brain-Computer Interface for In-Vehicle Driver Cognitive Load Measurement: Dataset and Baselines](https://mlsanity.com/paper/2304.04273/) (2023)

## Abstract (arXiv)

We exploit a self-supervised deep multi-task learning framework for electrocardiogram (ECG) -based emotion recognition. The proposed solution consists of two stages of learning a) learning ECG representations and b) learning to classify emotions. ECG representations are learned by a signal transformation recognition network. The network learns high-level abstract representations from unlabeled ECG data. Six different signal transformations are applied to the ECG signals, and transformation recognition is performed as pretext tasks. Training the model on pretext tasks helps the network learn spatiotemporal representations that generalize well across different datasets and different emotion categories. We transfer the weights of the self-supervised network to an emotion recognition network, where the convolutional layers are kept frozen and the dense layers are trained with labelled ECG data. We show that the proposed solution considerably improves the performance compared to a network trained using fully-supervised learning. New state-of-the-art results are set in classification of arousal, valence, affective states, and stress for the four utilized datasets. Extensive experiments are performed, providing interesting insights into the impact of using a multi-task self-supervised structure instead of a single-task model, as well as the optimum level of difficulty required for the pretext self-supervised tasks.