# SETransformer: A Hybrid Attention-Based Architecture for Robust Human Activity Recognition

arXiv: https://arxiv.org/abs/2505.19369 · HTML: https://mlsanity.com/paper/2505.19369/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 25 May 2025.

## Builds on (most cited first, 15 of 15)

- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Foundation Models for Time Series Analysis: A Tutorial and Survey](https://mlsanity.com/paper/2403.14735/) (2024)
- [UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understanding](https://mlsanity.com/paper/2308.11592/) (2023)
- [Towards Visual-Prompt Temporal Answering Grounding in Medical Instructional Video](https://mlsanity.com/paper/2203.06667/) (2022)
- [WcDT: World-centric Diffusion Transformer for Traffic Scene Generation](https://mlsanity.com/paper/2404.02082/) (2024)
- [Fully-Connected Spatial-Temporal Graph for Multivariate Time-Series Data](https://mlsanity.com/paper/2309.05305/) (2023)
- [Hang-Time HAR: A Benchmark Dataset for Basketball Activity Recognition using Wrist-Worn Inertial Sensors](https://mlsanity.com/paper/2305.13124/) (2023)
- [AHAR: Adaptive CNN for Energy-efficient Human Activity Recognition in Low-power Edge Devices](https://mlsanity.com/paper/2102.01875/) (2021)
- [Towards nation-wide analytical healthcare infrastructures: A privacy-preserving augmented knee rehabilitation case study](https://mlsanity.com/paper/2412.20733/) (2024)
- [Comet: A Communication-efficient and Performant Approximation for Private Transformer Inference](https://mlsanity.com/paper/2405.17485/) (2024)
- [Deep Learning for Computer Vision based Activity Recognition and Fall Detection of the Elderly: a Systematic Review](https://mlsanity.com/paper/2401.11790/) (2024)
- [Transformer-Based Approaches for Sensor-Based Human Activity Recognition: Opportunities and Challenges](https://mlsanity.com/paper/2410.13605/) (2024)
- [Spiking Neural Networks with Temporal Attention-Guided Adaptive Fusion for imbalanced Multi-modal Learning](https://mlsanity.com/paper/2505.14535/) (2025)

## Built on by (most cited first, 19 of 19)

- [Gated Multimodal Graph Learning for Personalized Recommendation](https://mlsanity.com/paper/2506.00107/) (2025)
- [Spec-LLaVA: Accelerating Vision-Language Models with Dynamic Tree-Based Speculative Decoding](https://mlsanity.com/paper/2509.11961/) (2025)
- [CenterMamba-SAM: Center-Prioritized Scanning and Temporal Prototypes for Brain Lesion Segmentation](https://mlsanity.com/paper/2511.01243/) (2025)
- [CATCH: A Modular Cross-domain Adaptive Template with Hook](https://mlsanity.com/paper/2510.26582/) (2025)
- [A Narrative Review on Large AI Models in Lung Cancer Screening, Diagnosis, and Treatment Planning](https://mlsanity.com/paper/2506.07236/) (2025)
- [Robust Anomaly Detection in Network Traffic: Evaluating Machine Learning Models on CICIDS2017](https://mlsanity.com/paper/2506.19877/) (2025)
- [Language-Driven Coordination and Learning in Multi-Agent Simulation Environments](https://mlsanity.com/paper/2506.04251/) (2025)
- [Analysis of Anonymous User Interaction Relationships and Prediction of Advertising Feedback Based on Graph Neural Network](https://mlsanity.com/paper/2506.13787/) (2025)
- [A Novel Deep Hybrid Framework with Ensemble-Based Feature Optimization for Robust Real-Time Human Activity Recognition](https://mlsanity.com/paper/2508.18695/) (2025)
- [Fatigue-Aware Adaptive Interfaces for Wearable Devices Using Deep Learning](https://mlsanity.com/paper/2506.13203/) (2025)
- [Enhanced Pre-training of Graph Neural Networks for Million-Scale Heterogeneous Graphs](https://mlsanity.com/paper/2510.12401/) (2025)
- [LPCAN: Lightweight Pyramid Cross-Attention Network for Rail Surface Defect Detection Using RGB-D Data](https://mlsanity.com/paper/2601.09118/) (2026)
- [Knowledge-Embedded and Hypernetwork-Guided Few-Shot Substation Meter Defect Image Generation Method](https://mlsanity.com/paper/2601.09238/) (2026)
- [MA-LipNet: Multi-Dimensional Attention Networks for Robust Lipreading](https://mlsanity.com/paper/2601.20881/) (2026)
- [The Diffusion Duet: Harmonizing Dual Channels with Wavelet Suppression for Image Separation](https://mlsanity.com/paper/2602.13361/) (2026)
- [Detecting Deepfakes with Multivariate Soft Blending and CLIP-based Image-Text Alignment](https://mlsanity.com/paper/2602.15903/) (2026)
- [Efficient Image Super-Resolution with Multi-Scale Spatial Adaptive Attention Networks](https://mlsanity.com/paper/2603.00118/) (2026)
- [A Deformable Attention-Based Detection Transformer with Cross-Scale Feature Fusion for Industrial Coil Spring Inspection](https://mlsanity.com/paper/2603.13429/) (2026)
- [Transmittance-Guided Structure-Texture Decomposition for Nighttime Image Dehazing](https://mlsanity.com/paper/2603.29507/) (2026)

## Abstract (arXiv)

Human Activity Recognition (HAR) using wearable sensor data has become a central task in mobile computing, healthcare, and human-computer interaction. Despite the success of traditional deep learning models such as CNNs and RNNs, they often struggle to capture long-range temporal dependencies and contextual relevance across multiple sensor channels. To address these limitations, we propose SETransformer, a hybrid deep neural architecture that combines Transformer-based temporal modeling with channel-wise squeeze-and-excitation (SE) attention and a learnable temporal attention pooling mechanism. The model takes raw triaxial accelerometer data as input and leverages global self-attention to capture activity-specific motion dynamics over extended time windows, while adaptively emphasizing informative sensor channels and critical time steps. We evaluate SETransformer on the WISDM dataset and demonstrate that it significantly outperforms conventional models including LSTM, GRU, BiLSTM, and CNN baselines. The proposed model achieves a validation accuracy of 84.68% and a macro F1-score of 84.64%, surpassing all baseline architectures by a notable margin. Our results show that SETransformer is a competitive and interpretable solution for real-world HAR tasks, with strong potential for deployment in mobile and ubiquitous sensing applications.