# Deep ConvLSTM with self-attention for human activity decoding using wearables

arXiv: https://arxiv.org/abs/2005.00698 · HTML: https://mlsanity.com/paper/2005.00698/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2020.

## Built on by (most cited first, 23 of 23)

- [Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey](https://mlsanity.com/paper/2302.02515/) (2023)
- [Cross-modal Knowledge Distillation for Vision-to-Sensor Action Recognition](https://mlsanity.com/paper/2112.01849/) (2021)
- [IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition](https://mlsanity.com/paper/2402.01049/) (2024)
- [OpenPack: A Large-scale Dataset for Recognizing Packaging Works in IoT-enabled Logistic Environments](https://mlsanity.com/paper/2212.11152/) (2022)
- [FLAME: Federated Learning Across Multi-device Environments](https://mlsanity.com/paper/2202.08922/) (2022)
- [Progressive Cross-modal Knowledge Distillation for Human Action Recognition](https://mlsanity.com/paper/2208.08090/) (2022)
- [Eat-Radar: Continuous Fine-Grained Intake Gesture Detection Using FMCW Radar and 3D Temporal Convolutional Network with Attention](https://mlsanity.com/paper/2211.04253/) (2022)
- [MAC-Gaze: Motion-Aware Continual Calibration for Mobile Gaze Tracking](https://mlsanity.com/paper/2505.22769/) (2025)
- [Multi-channel Time Series Decomposition Network For Generalizable Sensor-Based Activity Recognition](https://mlsanity.com/paper/2406.16872/) (2024)
- [OpenMarcie: Dataset for Multimodal Action Recognition in Industrial Environments](https://mlsanity.com/paper/2603.02390/) (2026)
- [SigFormer: Sparse Signal-Guided Transformer for Multi-Modal Human Action Segmentation](https://mlsanity.com/paper/2311.17428/) (2023)
- [GraMFedDHAR: Graph Based Multimodal Differentially Private Federated HAR](https://mlsanity.com/paper/2509.05671/) (2025)
- [Hi-OSCAR: Hierarchical Open-set Classifier for Human Activity Recognition](https://mlsanity.com/paper/2510.08635/) (2025)
- [Information Theory Inspired Pattern Analysis for Time-series Data](https://mlsanity.com/paper/2302.11654/) (2023)
- [Unified Contrastive Fusion Transformer for Multimodal Human Action Recognition](https://mlsanity.com/paper/2309.05032/) (2023)
- [Time Series Representation Models](https://mlsanity.com/paper/2405.18165/) (2024)
- [ST-ProC: A Graph-Prototypical Framework for Robust Semi-Supervised Travel Mode Identification](https://mlsanity.com/paper/2511.13702/) (2025)
- [Forensic Activity Classification Using Digital Traces from iPhones: A Machine Learning-based Approach](https://mlsanity.com/paper/2512.03786/) (2025)
- [Multilevel neural networks with dual-stage feature fusion for human activity recognition](https://mlsanity.com/paper/2604.16577/) (2026)
- [Leveraging Imperfect Medical Data: A Manifold-Consistent Spatio-Temporal Network for Sensor-based Human Activity Recognition](https://mlsanity.com/paper/2605.00913/) (2026)
- [EduGage: A Multimodal Dataset and Benchmark for Sensor-Based Momentary Assessment of Engagement in Self-Guided Video Learning](https://mlsanity.com/paper/2605.01238/) (2026)
- [HealthCAT: An Interpretable Encoder-only Transformer Framework for Health Indicator Prediction and Temporal Interpretation of Wearable Sensor Data](https://mlsanity.com/paper/2607.27635/) (2026)
- [Diffusion-Based Synthetic Data Pretraining for Enhancing Activity Recognition](https://mlsanity.com/paper/2610.02292/) (2026)

## Abstract (arXiv)

Decoding human activity accurately from wearable sensors can aid in applications related to healthcare and context awareness. The present approaches in this domain use recurrent and/or convolutional models to capture the spatio-temporal features from time-series data from multiple sensors. We propose a deep neural network architecture that not only captures the spatio-temporal features of multiple sensor time-series data but also selects, learns important time points by utilizing a self-attention mechanism. We show the validity of the proposed approach across different data sampling strategies on six public datasets and demonstrate that the self-attention mechanism gave a significant improvement in performance over deep networks using a combination of recurrent and convolution networks. We also show that the proposed approach gave a statistically significant performance enhancement over previous state-of-the-art methods for the tested datasets. The proposed methods open avenues for better decoding of human activity from multiple body sensors over extended periods of time. The code implementation for the proposed model is available at https://github.com/isukrit/encodingHumanActivity.