# Human Activity Recognition from Wearable Sensor Data Using Self-Attention

arXiv: https://arxiv.org/abs/2003.09018 · HTML: https://mlsanity.com/paper/2003.09018/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2020.

## Built on by (most cited first, 27 of 27)

- [Past, Present, and Future of Sensor-Based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task](https://mlsanity.com/paper/2411.14452/) (2024)
- [Learning from the Best: Contrastive Representations Learning Across Sensor Locations for Wearable Activity Recognition](https://mlsanity.com/paper/2210.01459/) (2022)
- [Adversarial Deep Feature Extraction Network for User Independent Human Activity Recognition](https://mlsanity.com/paper/2110.12163/) (2021)
- [TASKED: Transformer-based Adversarial learning for human activity recognition using wearable sensors via Self-KnowledgE Distillation](https://mlsanity.com/paper/2209.09092/) (2022)
- [Contrastive Learning with Cross-Modal Knowledge Mining for Multimodal Human Activity Recognition](https://mlsanity.com/paper/2205.10071/) (2022)
- [MicroNAS: Memory and Latency Constrained Hardware-Aware Neural Architecture Search for Time Series Classification on Microcontrollers](https://mlsanity.com/paper/2310.18384/) (2023)
- [Temporal Feature Alignment in Contrastive Self-Supervised Learning for Human Activity Recognition](https://mlsanity.com/paper/2210.03382/) (2022)
- [Multimodal Contrastive Learning with Hard Negative Sampling for Human Activity Recognition](https://mlsanity.com/paper/2309.01262/) (2023)
- [Transformer-Based Approaches for Sensor-Based Human Activity Recognition: Opportunities and Challenges](https://mlsanity.com/paper/2410.13605/) (2024)
- [Hierarchical Self Attention Based Autoencoder for Open-Set Human Activity Recognition](https://mlsanity.com/paper/2103.04279/) (2021)
- [Disentangling Imperfect: A Wavelet-Infused Multilevel Heterogeneous Network for Human Activity Recognition in Flawed Wearable Sensor Data](https://mlsanity.com/paper/2402.09434/) (2024)
- [Transformer Networks for Data Augmentation of Human Physical Activity Recognition](https://mlsanity.com/paper/2109.01081/) (2021)
- [Decomposing and Fusing Intra- and Inter-Sensor Spatio-Temporal Signal for Multi-Sensor Wearable Human Activity Recognition](https://mlsanity.com/paper/2501.10917/) (2025)
- [MHARFedLLM: Multimodal Human Activity Recognition Using Federated Large Language Model](https://mlsanity.com/paper/2508.01701/) (2025)
- [WaveGlove: Transformer-based hand gesture recognition using multiple inertial sensors](https://mlsanity.com/paper/2105.01753/) (2021)
- [Passive and Active Learning of Driver Behavior from Electric Vehicles](https://mlsanity.com/paper/2203.02179/) (2022)
- [Reducing Label Dependency in Human Activity Recognition with Wearables: From Supervised Learning to Novel Weakly Self-Supervised Approaches](https://mlsanity.com/paper/2512.19713/) (2025)
- [TinyGaze: Lightweight Gaze-Gesture Recognition on Commodity Mobile Devices](https://mlsanity.com/paper/2604.09658/) (2026)
- [STLGRU: Spatio-Temporal Lightweight Graph GRU for Traffic Flow Prediction](https://mlsanity.com/paper/2212.04548/) (2022)
- [DETACH : Decomposed Spatio-Temporal Alignment for Exocentric Video and Ambient Sensors with Staged Learning](https://mlsanity.com/paper/2512.20409/) (2025)
- [Triple Spectral Fusion for Sensor-based Human Activity Recognition](https://mlsanity.com/paper/2605.02743/) (2026)
- [WHAR Arena: Benchmarking the State of the Art in Efficient Wearable Human Activity Recognition](https://mlsanity.com/paper/2606.13194/) (2026)
- [Memory-Augmented LSTM Autoencoder for Unsupervised Activity Recognition with IMU Sensor Fusion](https://mlsanity.com/paper/2606.28377/) (2026)
- [Structure over Depth: A Single-Block Spatio-Temporal Transformer for Multi-Entity Reasoning](https://mlsanity.com/paper/2607.23077/) (2026)
- [Rotation-Invariant Multi-IMU Activity Recognition under Independent Per-Location Orientation Shifts](https://mlsanity.com/paper/2608.15621/) (2026)
- [Diffusion-Based Synthetic Data Pretraining for Enhancing Activity Recognition](https://mlsanity.com/paper/2610.02292/) (2026)
- [Characterizing the Performance Gap in Human Activity Recognition for Older Adults](https://mlsanity.com/paper/2610.02711/) (2026)

## Abstract (arXiv)

Human Activity Recognition from body-worn sensor data poses an inherent challenge in capturing spatial and temporal dependencies of time-series signals. In this regard, the existing recurrent or convolutional or their hybrid models for activity recognition struggle to capture spatio-temporal context from the feature space of sensor reading sequence. To address this complex problem, we propose a self-attention based neural network model that foregoes recurrent architectures and utilizes different types of attention mechanisms to generate higher dimensional feature representation used for classification. We performed extensive experiments on four popular publicly available HAR datasets: PAMAP2, Opportunity, Skoda and USC-HAD. Our model achieve significant performance improvement over recent state-of-the-art models in both benchmark test subjects and Leave-one-subject-out evaluation. We also observe that the sensor attention maps produced by our model is able capture the importance of the modality and placement of the sensors in predicting the different activity classes.