# Improving Deep Learning for HAR with shallow LSTMs

arXiv: https://arxiv.org/abs/2108.00702 · HTML: https://mlsanity.com/paper/2108.00702/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 44 references that are in the map.
- First posted on arXiv 2 Aug 2021.

## Builds on (most cited first, 12 of 12)

- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Visualizing and Understanding Recurrent Networks](https://mlsanity.com/paper/1506.02078/) (2015)
- [Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation](https://mlsanity.com/paper/1406.2984/) (2014)
- [Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities](https://mlsanity.com/paper/2001.07416/) (2020)
- [Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables](https://mlsanity.com/paper/1604.08880/) (2016)
- [DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing](https://mlsanity.com/paper/1611.01942/) (2016)
- [On Using Very Large Target Vocabulary for Neural Machine Translation](https://mlsanity.com/paper/1412.2007/) (2014)
- [Ensembles of Deep LSTM Learners for Activity Recognition using Wearables](https://mlsanity.com/paper/1703.09370/) (2017)
- [Question Answering with Subgraph Embeddings](https://mlsanity.com/paper/1406.3676/) (2014)
- [A C-LSTM Neural Network for Text Classification](https://mlsanity.com/paper/1511.08630/) (2015)
- [On Attention Models for Human Activity Recognition](https://mlsanity.com/paper/1805.07648/) (2018)
- [IMUTube: Automatic Extraction of Virtual on-body Accelerometry from Video for Human Activity Recognition](https://mlsanity.com/paper/2006.05675/) (2020)

## Built on by (most cited first, 30 of 30)

- [WEAR: An Outdoor Sports Dataset for Wearable and Egocentric Activity Recognition](https://mlsanity.com/paper/2304.05088/) (2023)
- [Masked Video and Body-worn IMU Autoencoder for Egocentric Action Recognition](https://mlsanity.com/paper/2407.06628/) (2024)
- [Past, Present, and Future of Sensor-Based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task](https://mlsanity.com/paper/2411.14452/) (2024)
- [Hang-Time HAR: A Benchmark Dataset for Basketball Activity Recognition using Wrist-Worn Inertial Sensors](https://mlsanity.com/paper/2305.13124/) (2023)
- [A Survey on Multimodal Wearable Sensor-based Human Action Recognition](https://mlsanity.com/paper/2404.15349/) (2024)
- [Temporal Action Localization for Inertial-based Human Activity Recognition](https://mlsanity.com/paper/2311.15831/) (2023)
- [TSAK: Two-Stage Semantic-Aware Knowledge Distillation for Efficient Wearable Modality and Model Optimization in Manufacturing Lines](https://mlsanity.com/paper/2408.14146/) (2024)
- [Self-Supervised Human Activity Recognition with Localized Time-Frequency Contrastive Representation Learning](https://mlsanity.com/paper/2209.00990/) (2022)
- [A Matter of Annotation: An Empirical Study on In Situ and Self-Recall Activity Annotations from Wearable Sensors](https://mlsanity.com/paper/2305.08752/) (2023)
- [P2LHAP:Wearable sensor-based human activity recognition, segmentation and forecast through Patch-to-Label Seq2Seq Transformer](https://mlsanity.com/paper/2403.08214/) (2024)
- [Scaling laws in wearable human activity recognition](https://mlsanity.com/paper/2502.03364/) (2025)
- [Weak-Annotation of HAR Datasets using Vision Foundation Models](https://mlsanity.com/paper/2408.05169/) (2024)
- [FedOpenHAR: Federated Multi-Task Transfer Learning for Sensor-Based Human Activity Recognition](https://mlsanity.com/paper/2311.07765/) (2023)
- [Deconfounding Causal Inference through Two-Branch Framework with Early-Forking for Sensor-Based Cross-Domain Activity Recognition](https://mlsanity.com/paper/2507.03898/) (2025)
- [randomHAR: Improving Ensemble Deep Learners for Human Activity Recognition with Sensor Selection and Reinforcement Learning](https://mlsanity.com/paper/2307.07770/) (2023)
- [Game of LLMs: Discovering Structural Constructs in Activities using Large Language Models](https://mlsanity.com/paper/2406.13777/) (2024)
- [FedBChain: A Blockchain-enabled Federated Learning Framework for Improving DeepConvLSTM with Comparative Strategy Insights](https://mlsanity.com/paper/2407.21282/) (2024)
- [MARAuder's Map: Motion-Aware Real-time Activity Recognition with Layout-Based Trajectories](https://mlsanity.com/paper/2511.05773/) (2025)
- [HARMES: A Multi-Modal Dataset for Wearable Human Activity Recognition with Motion, Environmental Sensing and Sound](https://mlsanity.com/paper/2605.02596/) (2026)
- [Tutorial on Deep Learning for Human Activity Recognition](https://mlsanity.com/paper/2110.06663/) (2021)
- [Enhancing Causal Estimation through Unlabeled Offline Data](https://mlsanity.com/paper/2202.07895/) (2022)
- [Distilled Mid-Fusion Transformer Networks for Multi-Modal Human Activity Recognition](https://mlsanity.com/paper/2305.03810/) (2023)
- [In Shift and In Variance: Assessing the Robustness of HAR Deep Learning Models against Variability](https://mlsanity.com/paper/2503.11466/) (2025)
- [Detecting Informative Channels: ActionFormer](https://mlsanity.com/paper/2505.20739/) (2025)
- [DeepConvContext: A Multi-Scale Approach to Timeseries Classification in Human Activity Recognition](https://mlsanity.com/paper/2505.20894/) (2025)
- [Knowledge Distillation for Reservoir-based Classifier: Human Activity Recognition](https://mlsanity.com/paper/2505.22985/) (2025)
- [MicroBi-ConvLSTM: An Ultra-Lightweight Efficient Model for Human Activity Recognition on Resource Constrained Devices](https://mlsanity.com/paper/2602.06523/) (2026)
- [BabyMamba-HAR: Lightweight Selective State Space Models for Efficient Human Activity Recognition on Resource Constrained Devices](https://mlsanity.com/paper/2602.09872/) (2026)
- [WHAR Arena: Benchmarking the State of the Art in Efficient Wearable Human Activity Recognition](https://mlsanity.com/paper/2606.13194/) (2026)
- [RevalExo: A Functional Daily-Activity Benchmark for Inertial and Visual Locomotion Mode Recognition in Older Adults and Clinical Cohorts](https://mlsanity.com/paper/2609.08090/) (2026)

## Abstract (arXiv)

Recent studies in Human Activity Recognition (HAR) have shown that Deep Learning methods are able to outperform classical Machine Learning algorithms. One popular Deep Learning architecture in HAR is the DeepConvLSTM. In this paper we propose to alter the DeepConvLSTM architecture to employ a 1-layered instead of a 2-layered LSTM. We validate our architecture change on 5 publicly available HAR datasets by comparing the predictive performance with and without the change employing varying hidden units within the LSTM layer(s). Results show that across all datasets, our architecture consistently improves on the original one: Recognition performance increases up to 11.7% for the F1-score, and our architecture significantly decreases the amount of learnable parameters. This improvement over DeepConvLSTM decreases training time by as much as 48%. Our results stand in contrast to the belief that one needs at least a 2-layered LSTM when dealing with sequential data. Based on our results we argue that said claim might not be applicable to sensor-based HAR.