# PhysioSync: Temporal and Cross-Modal Contrastive Learning Inspired by Physiological Synchronization for EEG-Based Emotion Recognition

arXiv: https://arxiv.org/abs/2504.17163 · HTML: https://mlsanity.com/paper/2504.17163/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 79 references that are in the map.
- First posted on arXiv 24 Apr 2025.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [Cross Attention Network for Few-shot Classification](https://mlsanity.com/paper/1910.07677/) (2019)
- [Attentional Feature Fusion](https://mlsanity.com/paper/2009.14082/) (2020)
- [Contrastive Learning of Subject-Invariant EEG Representations for Cross-Subject Emotion Recognition](https://mlsanity.com/paper/2109.09559/) (2021)
- [TSception: Capturing Temporal Dynamics and Spatial Asymmetry from EEG for Emotion Recognition](https://mlsanity.com/paper/2104.02935/) (2021)
- [CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding](https://mlsanity.com/paper/2010.08670/) (2020)
- [Data Augmentation techniques in time series domain: A survey and taxonomy](https://mlsanity.com/paper/2206.13508/) (2022)
- [Semi-Supervised Dual-Stream Self-Attentive Adversarial Graph Contrastive Learning for Cross-Subject EEG-based Emotion Recognition](https://mlsanity.com/paper/2308.11635/) (2023)
- [Hypercomplex Multimodal Emotion Recognition from EEG and Peripheral Physiological Signals](https://mlsanity.com/paper/2310.07648/) (2023)
- [Self-supervised Group Meiosis Contrastive Learning for EEG-Based Emotion Recognition](https://mlsanity.com/paper/2208.00877/) (2022)
- [Joint Contrastive Learning with Feature Alignment for Cross-Corpus EEG-based Emotion Recognition](https://mlsanity.com/paper/2404.09559/) (2024)
- [TACOformer:Token-channel compounded Cross Attention for Multimodal Emotion Recognition](https://mlsanity.com/paper/2306.13592/) (2023)

## Built on by (most cited first, 3 of 3)

- [FedMultiEmo: Real-Time Emotion Recognition via Multimodal Federated Learning](https://mlsanity.com/paper/2507.15470/) (2025)
- [Multimodal Functional Maximum Correlation for Emotion Recognition](https://mlsanity.com/paper/2512.23076/) (2025)
- [Cross-Subject EEG Emotion Recognition Based on Temporal Asynchronous Alignment Contrastive Learning](https://mlsanity.com/paper/2605.22379/) (2026)

## Abstract (arXiv)

Electroencephalography (EEG) signals provide a promising and involuntary reflection of brain activity related to emotional states, offering significant advantages over behavioral cues like facial expressions. However, EEG signals are often noisy, affected by artifacts, and vary across individuals, complicating emotion recognition. While multimodal approaches have used Peripheral Physiological Signals (PPS) like GSR to complement EEG, they often overlook the dynamic synchronization and consistent semantics between the modalities. Additionally, the temporal dynamics of emotional fluctuations across different time resolutions in PPS remain underexplored. To address these challenges, we propose PhysioSync, a novel pre-training framework leveraging temporal and cross-modal contrastive learning, inspired by physiological synchronization phenomena. PhysioSync incorporates Cross-Modal Consistency Alignment (CM-CA) to model dynamic relationships between EEG and complementary PPS, enabling emotion-related synchronizations across modalities. Besides, it introduces Long- and Short-Term Temporal Contrastive Learning (LS-TCL) to capture emotional synchronization at different temporal resolutions within modalities. After pre-training, cross-resolution and cross-modal features are hierarchically fused and fine-tuned to enhance emotion recognition. Experiments on DEAP and DREAMER datasets demonstrate PhysioSync's advanced performance under uni-modal and cross-modal conditions, highlighting its effectiveness for EEG-centered emotion recognition.