# Fusion of Physiological and Behavioural Signals on SPD Manifolds with Application to Stress and Pain Detection

arXiv: https://arxiv.org/abs/2207.08811 · HTML: https://mlsanity.com/paper/2207.08811/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 32 references that are in the map.
- First posted on arXiv 17 Jul 2022.

## Builds on (most cited first, 6 of 6)

- [Self-supervised ECG Representation Learning for Emotion Recognition](https://mlsanity.com/paper/2002.03898/) (2020)
- [Kernel Methods on Riemannian Manifolds with Gaussian RBF Kernels](https://mlsanity.com/paper/1412.0265/) (2014)
- [StressNAS: Affect State and Stress Detection Using Neural Architecture Search](https://mlsanity.com/paper/2108.12502/) (2021)
- [Automatic Estimation of Self-Reported Pain by Interpretable Representations of Motion Dynamics](https://mlsanity.com/paper/2006.13882/) (2020)
- [Automatic Analysis of Facial Expressions Based on Deep Covariance Trajectories](https://mlsanity.com/paper/1810.11392/) (2018)
- [Log-Euclidean Bag of Words for Human Action Recognition](https://mlsanity.com/paper/1406.2139/) (2014)

## Built on by (most cited first, 2 of 2)

- [A Multimodal Intermediate Fusion Network with Manifold Learning for Stress Detection](https://mlsanity.com/paper/2403.08077/) (2024)
- [RepSPD: Enhancing SPD Manifold Representation in EEGs via Dynamic Graphs](https://mlsanity.com/paper/2602.22981/) (2026)

## Abstract (arXiv)

Existing multimodal stress/pain recognition approaches generally extract features from different modalities independently and thus ignore cross-modality correlations. This paper proposes a novel geometric framework for multimodal stress/pain detection utilizing Symmetric Positive Definite (SPD) matrices as a representation that incorporates the correlation relationship of physiological and behavioural signals from covariance and cross-covariance. Considering the non-linearity of the Riemannian manifold of SPD matrices, well-known machine learning techniques are not suited to classify these matrices. Therefore, a tangent space mapping method is adopted to map the derived SPD matrix sequences to the vector sequences in the tangent space where the LSTM-based network can be applied for classification. The proposed framework has been evaluated on two public multimodal datasets, achieving both the state-of-the-art results for stress and pain detection tasks.