# "Reading Between the Heat": Co-Teaching Body Thermal Signatures for Non-intrusive Stress Detection

arXiv: https://arxiv.org/abs/2310.09932 · HTML: https://mlsanity.com/paper/2310.09932/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 126 references that are in the map.
- First posted on arXiv 15 Oct 2023.

## Builds on (most cited first, 23 of 23)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Entropy-SGD: Biasing Gradient Descent Into Wide Valleys](https://mlsanity.com/paper/1611.01838/) (2016)
- [Omni-Scale Feature Learning for Person Re-Identification](https://mlsanity.com/paper/1905.00953/) (2019)
- [DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution](https://mlsanity.com/paper/2006.02334/) (2020)
- [Qualitatively characterizing neural network optimization problems](https://mlsanity.com/paper/1412.6544/) (2014)
- [Generalization in Deep Learning](https://mlsanity.com/paper/1710.05468/) (2017)
- [Feature Erasing and Diffusion Network for Occluded Person Re-Identification](https://mlsanity.com/paper/2112.08740/) (2021)
- [Torchreid: A Library for Deep Learning Person Re-Identification in Pytorch](https://mlsanity.com/paper/1910.10093/) (2019)
- [VALHALLA: Visual Hallucination for Machine Translation](https://mlsanity.com/paper/2206.00100/) (2022)
- [An empirical analysis of the optimization of deep network loss surfaces](https://mlsanity.com/paper/1612.04010/) (2016)
- [Instant Automated Inference of Perceived Mental Stress through Smartphone PPG and Thermal Imaging](https://mlsanity.com/paper/1901.00449/) (2018)
- [An Ethical Highlighter for People-Centric Dataset Creation](https://mlsanity.com/paper/2011.13583/) (2020)
- [Robust Latent Representations via Cross-Modal Translation and Alignment](https://mlsanity.com/paper/2011.01631/) (2020)
- [StressNet: Detecting Stress in Thermal Videos](https://mlsanity.com/paper/2011.09540/) (2020)
- [Psychophysiological Arousal in Young Children Who Stutter: An Interpretable AI Approach](https://mlsanity.com/paper/2208.08859/) (2022)
- [Employing Multimodal Machine Learning for Stress Detection](https://mlsanity.com/paper/2306.09385/) (2023)
- [Physiological and Affective Computing through Thermal Imaging: A Survey](https://mlsanity.com/paper/1908.10307/) (2019)
- [Privacy-Preserving Video Conferencing via Thermal-Generative Images](https://mlsanity.com/paper/2303.09279/) (2023)

## Built on by (most cited first, 2 of 2)

- [CurvFed: Curvature-Aligned Federated Learning for Fairness without Demographics](https://mlsanity.com/paper/2404.19725/) (2024)
- [CRoP: Context-wise Robust Static Human-Sensing Personalization](https://mlsanity.com/paper/2409.17994/) (2024)

## Abstract (arXiv)

Stress impacts our physical and mental health as well as our social life. A passive and contactless indoor stress monitoring system can unlock numerous important applications such as workplace productivity assessment, smart homes, and personalized mental health monitoring. While the thermal signatures from a user's body captured by a thermal camera can provide important information about the "fight-flight" response of the sympathetic and parasympathetic nervous system, relying solely on thermal imaging for training a stress prediction model often lead to overfitting and consequently a suboptimal performance. This paper addresses this challenge by introducing ThermaStrain, a novel co-teaching framework that achieves high-stress prediction performance by transferring knowledge from the wearable modality to the contactless thermal modality. During training, ThermaStrain incorporates a wearable electrodermal activity (EDA) sensor to generate stress-indicative representations from thermal videos, emulating stress-indicative representations from a wearable EDA sensor. During testing, only thermal sensing is used, and stress-indicative patterns from thermal data and emulated EDA representations are extracted to improve stress assessment. The study collected a comprehensive dataset with thermal video and EDA data under various stress conditions and distances. ThermaStrain achieves an F1 score of 0.8293 in binary stress classification, outperforming the thermal-only baseline approach by over 9%. Extensive evaluations highlight ThermaStrain's effectiveness in recognizing stress-indicative attributes, its adaptability across distances and stress scenarios, real-time executability on edge platforms, its applicability to multi-individual sensing, ability to function on limited visibility and unfamiliar conditions, and the advantages of its co-teaching approach.