# Hierarchical Self Attention Based Autoencoder for Open-Set Human Activity Recognition

arXiv: https://arxiv.org/abs/2103.04279 · HTML: https://mlsanity.com/paper/2103.04279/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 30 references that are in the map.
- First posted on arXiv 7 Mar 2021.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [C2AE: Class Conditioned Auto-Encoder for Open-set Recognition](https://mlsanity.com/paper/1904.01198/) (2019)
- [Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables](https://mlsanity.com/paper/1604.08880/) (2016)
- [Ensembles of Deep LSTM Learners for Activity Recognition using Wearables](https://mlsanity.com/paper/1703.09370/) (2017)
- [On Attention Models for Human Activity Recognition](https://mlsanity.com/paper/1805.07648/) (2018)
- [Understanding and Improving Recurrent Networks for Human Activity Recognition by Continuous Attention](https://mlsanity.com/paper/1810.04038/) (2018)
- [Human Activity Recognition from Wearable Sensor Data Using Self-Attention](https://mlsanity.com/paper/2003.09018/) (2020)
- [q-Space Novelty Detection with Variational Autoencoders](https://mlsanity.com/paper/1806.02997/) (2018)
- [Interpretable Spatio-temporal Attention for Video Action Recognition](https://mlsanity.com/paper/1810.04511/) (2018)
- [A Hierarchical Self-Attentive Model for Recommending User-Generated Item Lists](https://mlsanity.com/paper/1912.13023/) (2019)

## Built on by (most cited first, 3 of 3)

- [Decomposing and Fusing Intra- and Inter-Sensor Spatio-Temporal Signal for Multi-Sensor Wearable Human Activity Recognition](https://mlsanity.com/paper/2501.10917/) (2025)
- [Hi-OSCAR: Hierarchical Open-set Classifier for Human Activity Recognition](https://mlsanity.com/paper/2510.08635/) (2025)
- [PovNet+: A Deep Learning Architecture for Socially Assistive Robots to Learn and Assist with Multiple Activities of Daily Living](https://mlsanity.com/paper/2602.00131/) (2026)

## Abstract (arXiv)

Wearable sensor based human activity recognition is a challenging problem due to difficulty in modeling spatial and temporal dependencies of sensor signals. Recognition models in closed-set assumption are forced to yield members of known activity classes as prediction. However, activity recognition models can encounter an unseen activity due to body-worn sensor malfunction or disability of the subject performing the activities. This problem can be addressed through modeling solution according to the assumption of open-set recognition. Hence, the proposed self attention based approach combines data hierarchically from different sensor placements across time to classify closed-set activities and it obtains notable performance improvement over state-of-the-art models on five publicly available datasets. The decoder in this autoencoder architecture incorporates self-attention based feature representations from encoder to detect unseen activity classes in open-set recognition setting. Furthermore, attention maps generated by the hierarchical model demonstrate explainable selection of features in activity recognition. We conduct extensive leave one subject out validation experiments that indicate significantly improved robustness to noise and subject specific variability in body-worn sensor signals. The source code is available at: github.com/saif-mahmud/hierarchical-attention-HAR