# Self-supervised Learning for ECG-based Emotion Recognition

arXiv: https://arxiv.org/abs/1910.07497 · HTML: https://mlsanity.com/paper/1910.07497/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2019.

## Built on by (most cited first, 30 of 32)

- [Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency](https://mlsanity.com/paper/2206.08496/) (2022)
- [Self-Supervised Representation Learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2110.09327/) (2021)
- [SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling](https://mlsanity.com/paper/2302.00861/) (2023)
- [Deep Gait Recognition: A Survey](https://mlsanity.com/paper/2102.09546/) (2021)
- [Soft Contrastive Learning for Time Series](https://mlsanity.com/paper/2312.16424/) (2023)
- [Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Prediction](https://mlsanity.com/paper/2102.10106/) (2021)
- [3KG: Contrastive Learning of 12-Lead Electrocardiograms using Physiologically-Inspired Augmentations](https://mlsanity.com/paper/2106.04452/) (2021)
- [A Transformer Architecture for Stress Detection from ECG](https://mlsanity.com/paper/2108.09737/) (2021)
- [Self-supervised Contrastive Learning for EEG-based Sleep Staging](https://mlsanity.com/paper/2109.07839/) (2021)
- [Self-supervised Contrastive Learning of Multi-view Facial Expressions](https://mlsanity.com/paper/2108.06723/) (2021)
- [Transformer-Based Self-Supervised Learning for Emotion Recognition](https://mlsanity.com/paper/2204.05103/) (2022)
- [A Survey on Physiological Signal Based Emotion Recognition](https://mlsanity.com/paper/2205.10466/) (2022)
- [Analysis of Augmentations for Contrastive ECG Representation Learning](https://mlsanity.com/paper/2206.07656/) (2022)
- [Spatiotemporal Contrastive Learning of Facial Expressions in Videos](https://mlsanity.com/paper/2108.03064/) (2021)
- [Deep Learning for Time Series Forecasting: A Survey](https://mlsanity.com/paper/2503.10198/) (2025)
- [Attentive Cross-modal Connections for Deep Multimodal Wearable-based Emotion Recognition](https://mlsanity.com/paper/2108.02241/) (2021)
- [Self-Supervised Audio-Visual Representation Learning with Relaxed Cross-Modal Synchronicity](https://mlsanity.com/paper/2111.05329/) (2021)
- [CREMA: A Contrastive Regularized Masked Autoencoder for Robust ECG Diagnostics across Clinical Domains](https://mlsanity.com/paper/2407.07110/) (2024)
- [A Simple Self-Supervised ECG Representation Learning Method via Manipulated Temporal-Spatial Reverse Detection](https://mlsanity.com/paper/2202.12458/) (2022)
- [In-Distribution and Out-of-Distribution Self-supervised ECG Representation Learning for Arrhythmia Detection](https://mlsanity.com/paper/2304.06427/) (2023)
- [AttX: Attentive Cross-Connections for Fusion of Wearable Signals in Emotion Recognition](https://mlsanity.com/paper/2206.04625/) (2022)
- [Self-supervised Group Meiosis Contrastive Learning for EEG-Based Emotion Recognition](https://mlsanity.com/paper/2208.00877/) (2022)
- [Emotion Recognition with Pre-Trained Transformers Using Multimodal Signals](https://mlsanity.com/paper/2212.13885/) (2022)
- [Self-Supervised Learning via multi-Transformation Classification for Action Recognition](https://mlsanity.com/paper/2102.10378/) (2021)
- [AVCAffe: A Large Scale Audio-Visual Dataset of Cognitive Load and Affect for Remote Work](https://mlsanity.com/paper/2205.06887/) (2022)
- [ProductGraphSleepNet: Sleep Staging using Product Spatio-Temporal Graph Learning with Attentive Temporal Aggregation](https://mlsanity.com/paper/2212.04881/) (2022)
- [Coherent, super resolved radar beamforming using self-supervised learning](https://mlsanity.com/paper/2106.13085/) (2021)
- [Remote Heart Rate Monitoring in Smart Environments from Videos with Self-supervised Pre-training](https://mlsanity.com/paper/2310.15388/) (2023)
- [Consistency Training of Multi-exit Architectures for Sensor Data](https://mlsanity.com/paper/2109.13192/) (2021)
- [A Self-Supervised Task for Fault Detection in Satellite Multivariate Time Series](https://mlsanity.com/paper/2407.02861/) (2024)

## Abstract (arXiv)

We present an electrocardiogram (ECG) -based emotion recognition system using self-supervised learning. Our proposed architecture consists of two main networks, a signal transformation recognition network and an emotion recognition network. First, unlabelled data are used to successfully train the former network to detect specific pre-determined signal transformations in the self-supervised learning step. Next, the weights of the convolutional layers of this network are transferred to the emotion recognition network, and two dense layers are trained in order to classify arousal and valence scores. We show that our self-supervised approach helps the model learn the ECG feature manifold required for emotion recognition, performing equal or better than the fully-supervised version of the model. Our proposed method outperforms the state-of-the-art in ECG-based emotion recognition with two publicly available datasets, SWELL and AMIGOS. Further analysis highlights the advantage of our self-supervised approach in requiring significantly less data to achieve acceptable results.