# PHemoNet: A Multimodal Network for Physiological Signals

arXiv: https://arxiv.org/abs/2410.00010 · HTML: https://mlsanity.com/paper/2410.00010/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 16 of 16)

- [Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameters](https://mlsanity.com/paper/2102.08597/) (2021)
- [A Survey of Complex-Valued Neural Networks](https://mlsanity.com/paper/2101.12249/) (2021)
- [Quaternion Convolutional Neural Networks for Heterogeneous Image Processing](https://mlsanity.com/paper/1811.02656/) (2018)
- [PHNNs: Lightweight Neural Networks via Parameterized Hypercomplex Convolutions](https://mlsanity.com/paper/2110.04176/) (2021)
- [Speech Emotion Recognition Using Quaternion Convolutional Neural Networks](https://mlsanity.com/paper/2111.00404/) (2021)
- [Quaternion-valued Correlation Learning for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2305.07283/) (2023)
- [Dual Quaternion Ambisonics Array for Six-Degree-of-Freedom Acoustic Representation](https://mlsanity.com/paper/2204.01851/) (2022)
- [Hypercomplex Multimodal Emotion Recognition from EEG and Peripheral Physiological Signals](https://mlsanity.com/paper/2310.07648/) (2023)
- [Learning Speech Emotion Representations in the Quaternion Domain](https://mlsanity.com/paper/2204.02385/) (2022)
- [Multi-View Hypercomplex Learning for Breast Cancer Screening](https://mlsanity.com/paper/2204.05798/) (2022)
- [Attention-Map Augmentation for Hypercomplex Breast Cancer Classification](https://mlsanity.com/paper/2310.07633/) (2023)
- [Generalizing Medical Image Representations via Quaternion Wavelet Networks](https://mlsanity.com/paper/2310.10224/) (2023)
- [Dual Quaternion Rotational and Translational Equivariance in 3D Rigid Motion Modelling](https://mlsanity.com/paper/2310.07623/) (2023)
- [Towards Explaining Hypercomplex Neural Networks](https://mlsanity.com/paper/2403.17929/) (2024)
- [PHYDI: Initializing Parameterized Hypercomplex Neural Networks as Identity Functions](https://mlsanity.com/paper/2310.07612/) (2023)
- [Efficient ECG-based Atrial Fibrillation Detection via Parameterised Hypercomplex Neural Networks](https://mlsanity.com/paper/2211.02678/) (2022)

## Built on by (most cited first, 2 of 2)

- [Guess What I Think: Streamlined EEG-to-Image Generation with Latent Diffusion Models](https://mlsanity.com/paper/2410.02780/) (2024)
- [BioKD: Selective Physiology-to-Video Knowledge Distillation via Reliability Gate for Emotion Recognition](https://mlsanity.com/paper/2608.06023/) (2026)

## Abstract (arXiv)

Emotion recognition is essential across numerous fields, including medical applications and brain-computer interface (BCI). Emotional responses include behavioral reactions, such as tone of voice and body movement, and changes in physiological signals, such as the electroencephalogram (EEG). The latter are involuntary, thus they provide a reliable input for identifying emotions, in contrast to the former which individuals can consciously control. These signals reveal true emotional states without intentional alteration, thus increasing the accuracy of emotion recognition models. However, multimodal deep learning methods from physiological signals have not been significantly investigated. In this paper, we introduce PHemoNet, a fully hypercomplex network for multimodal emotion recognition from physiological signals. In detail, the architecture comprises modality-specific encoders and a fusion module. Both encoders and fusion modules are defined in the hypercomplex domain through parameterized hypercomplex multiplications (PHMs) that can capture latent relations between the different dimensions of each modality and between the modalities themselves. The proposed method outperforms current state-of-the-art models on the MAHNOB-HCI dataset in classifying valence and arousal using electroencephalograms (EEGs) and peripheral physiological signals. The code for this work is available at https://github.com/ispamm/MHyEEG.