mlsanityData to 7 Oct 2026

Efficient-Husformer: Efficient Multimodal Transformer Hyperparameter Optimization for Stress and Cognitive Loads

arXiv 2511.22362 ↗First posted 27 Nov 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Transformer-based models have gained considerable attention in the field of physiological signal analysis. They leverage long-range dependencies and complex patterns in temporal signals, allowing them to achieve performance superior to traditional RNN and CNN models. However, they require high computational intensity and memory demands. In this work, we present Efficient-Husformer, a novel Transformer-based architecture developed with hyperparameter optimization (HPO) for multi-class stress detection across two multimodal physiological datasets (WESAD and CogLoad). The main contributions of this work are: (1) the design of a structured search space, targeting effective hyperparameter optimization; (2) a comprehensive ablation study evaluating the impact of architectural decisions; (3) consistent performance improvements over the original Husformer, with the best configuration achieving an accuracy of 88.41 and 92.61 (improvements of 13.83% and 6.98%) on WESAD and CogLoad datasets, respectively. The best-performing configuration is achieved with the (L + dm) or (L + FFN) modality combinations, using a single layer, 3 attention heads, a model dimension of 18/30, and FFN dimension of 120/30, resulting in a compact model with only about 30k parameters.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 35 references that are in the map. First posted on arXiv 27 Nov 2025.

Builds on

22 of 35 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  3. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  4. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
  5. 2020Low-Rank Bottleneck in Multi-head Attention Modelscited by 88
  6. 2020A Close Look at Deep Learning with Small Datacited by 54
  7. 2020On the Transformer Growth for Progressive BERT Trainingcited by 49
  8. 2022Staged Training for Transformer Language Modelscited by 49
  9. 2022Automated Progressive Learning for Efficient Training of Vision Transformerscited by 36
  10. 2021A Transformer Architecture for Stress Detection from ECGcited by 27
  11. 2017DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Frameworkcited by 22
  12. 2016Convolutional RNN: an Enhanced Model for Extracting Features from Sequential Datacited by 17
10 more
  1. 2022Transformer-Based Self-Supervised Learning for Emotion Recognitioncited by 16
  2. 2023One Wide Feedforward is All You Needcited by 15
  3. 2024Minusformer: Improving Time Series Forecasting by Progressively Learning Residualscited by 10
  4. 2024Hyperparameter Importance Analysis for Multi-Objective AutoMLcited by 9
  5. 2020Layer-wise training convolutional neural networks with smaller filters for human activity recognition using wearable sensorscited by 7
  6. 2022Husformer: A Multi-Modal Transformer for Multi-Modal Human State Recognitioncited by 7
  7. 2021MUSER: MUltimodal Stress Detection using Emotion Recognition as an Auxiliary Taskcited by 4
  8. 2024Efficient Training of Large Vision Models via Advanced Automated Progressive Learningcited by 4
  9. 2022Number of Attention Heads vs Number of Transformer-Encoders in Computer Visioncited by 2
  10. 2023Multi-scale Transformer-based Network for Emotion Recognition from Multi Physiological Signalscited by 1

Built on by

Not cited yet.