# Re-thinking Human Activity Recognition with Hierarchy-aware Label Relationship Modeling

arXiv: https://arxiv.org/abs/2403.05557 · HTML: https://mlsanity.com/paper/2403.05557/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 24 references that are in the map.
- First posted on arXiv 11 Feb 2024.

## Builds on (most cited first, 9 of 9)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Graph WaveNet for Deep Spatial-Temporal Graph Modeling](https://mlsanity.com/paper/1906.00121/) (2019)
- [Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text Classification](https://mlsanity.com/paper/2203.03825/) (2022)
- [Embedding Label Structures for Fine-Grained Feature Representation](https://mlsanity.com/paper/1512.02895/) (2015)
- [SMATE: Semi-Supervised Spatio-Temporal Representation Learning on Multivariate Time Series](https://mlsanity.com/paper/2110.00578/) (2021)
- [Graph Convolutional Networks for Traffic Forecasting with Missing Values](https://mlsanity.com/paper/2212.06419/) (2022)
- [Domain invariant hierarchical embedding for grocery products recognition](https://mlsanity.com/paper/1902.00760/) (2019)
- [On Handling Catastrophic Forgetting for Incremental Learning of Human Physical Activity on the Edge](https://mlsanity.com/paper/2302.09310/) (2023)

## Abstract (arXiv)

Human Activity Recognition (HAR) has been studied for decades, from data collection, learning models, to post-processing and result interpretations. However, the inherent hierarchy in the activities remains relatively under-explored, despite its significant impact on model performance and interpretation. In this paper, we propose H-HAR, by rethinking the HAR tasks from a fresh perspective by delving into their intricate global label relationships. Rather than building multiple classifiers separately for multi-layered activities, we explore the efficacy of a flat model enhanced with graph-based label relationship modeling. Being hierarchy-aware, the graph-based label modeling enhances the fundamental HAR model, by incorporating intricate label relationships into the model. We validate the proposal with a multi-label classifier on complex human activity data. The results highlight the advantages of the proposal, which can be vertically integrated into advanced HAR models to further enhance their performances.