# Instant Automated Inference of Perceived Mental Stress through Smartphone PPG and Thermal Imaging

arXiv: https://arxiv.org/abs/1901.00449 · HTML: https://mlsanity.com/paper/1901.00449/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2018.

## Built on by (most cited first, 12 of 12)

- [Self-supervised Contrastive Learning of Multi-view Facial Expressions](https://mlsanity.com/paper/2108.06723/) (2021)
- [FactorizePhys: Matrix Factorization for Multidimensional Attention in Remote Physiological Sensing](https://mlsanity.com/paper/2411.01542/) (2024)
- [Spatiotemporal Contrastive Learning of Facial Expressions in Videos](https://mlsanity.com/paper/2108.03064/) (2021)
- [Analysis of Semi-Supervised Methods for Facial Expression Recognition](https://mlsanity.com/paper/2208.00544/) (2022)
- [Rethinking Eye-blink: Assessing Task Difficulty through Physiological Representation of Spontaneous Blinking](https://mlsanity.com/paper/2102.06690/) (2021)
- [Self-adversarial Multi-scale Contrastive Learning for Semantic Segmentation of Thermal Facial Images](https://mlsanity.com/paper/2209.10700/) (2022)
- [Human Stress Assessment: A Comprehensive Review of Methods Using Wearable Sensors and Non-wearable Techniques](https://mlsanity.com/paper/2202.03033/) (2022)
- ["Reading Between the Heat": Co-Teaching Body Thermal Signatures for Non-intrusive Stress Detection](https://mlsanity.com/paper/2310.09932/) (2023)
- [Contrastive Learning of View-Invariant Representations for Facial Expressions Recognition](https://mlsanity.com/paper/2311.06852/) (2023)
- [CAREForMe: Contextual Multi-Armed Bandit Recommendation Framework for Mental Health](https://mlsanity.com/paper/2401.15188/) (2024)
- [Machine Learning Based Prediction of Future Stress Events in a Driving Scenario](https://mlsanity.com/paper/2106.07542/) (2021)
- [SFF-DA: Sptialtemporal Feature Fusion for Detecting Anxiety Nonintrusively](https://mlsanity.com/paper/2208.06411/) (2022)

## Abstract (arXiv)

Background: A smartphone is a promising tool for daily cardiovascular measurement and mental stress monitoring. A smartphone camera-based PhotoPlethysmoGraphy (PPG) and a low-cost thermal camera can be used to create cheap, convenient and mobile monitoring systems. However, to ensure reliable monitoring results, a person has to remain still for several minutes while a measurement is being taken. This is very cumbersome and makes its use in real-life mobile situations quite impractical. Objective: We propose a system which combines PPG and thermography with the aim of improving cardiovascular signal quality and capturing stress responses quickly. Methods: Using a smartphone camera with a low cost thermal camera added on, we built a novel system which continuously and reliably measures two different types of cardiovascular events: i) blood volume pulse and ii) vasoconstriction/dilation-induced temperature changes of the nose tip. 17 healthy participants, involved in a series of stress-inducing mental workload tasks, measured their physiological responses to stressors over a short window of time (20 seconds) immediately after each task. Participants reported their level of perceived mental stress using a 10-cm Visual Analogue Scale (VAS). We used normalized K-means clustering to reduce interpersonal differences in the self-reported ratings. For the instant stress inference task, we built novel low-level feature sets representing variability of cardiovascular patterns. We then used the automatic feature learning capability of artificial Neural Networks (NN) to improve the mapping between the extracted set of features and the self-reported ratings. We compared our proposed method with existing hand-engineered features-based machine learning methods. Results, Conclusions: ... due to limited space here, we refer to our manuscript.