# Fine-grained Human Activity Recognition Using Virtual On-body Acceleration Data

arXiv: https://arxiv.org/abs/2211.01342 · HTML: https://mlsanity.com/paper/2211.01342/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 36 references that are in the map.
- First posted on arXiv 2 Nov 2022.

## Builds on (most cited first, 5 of 5)

- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [A review of 3D human pose estimation algorithms for markerless motion capture](https://mlsanity.com/paper/2010.06449/) (2020)
- [IMUTube: Automatic Extraction of Virtual on-body Accelerometry from Video for Human Activity Recognition](https://mlsanity.com/paper/2006.05675/) (2020)
- [Exercise Motion Classification from Large-Scale Wearable Sensor Data Using Convolutional Neural Networks](https://mlsanity.com/paper/1610.07031/) (2016)

## Built on by (most cited first, 2 of 2)

- [On the Benefit of Generative Foundation Models for Human Activity Recognition](https://mlsanity.com/paper/2310.12085/) (2023)
- [Improvement of human health lifespan with hybrid group pose estimation methods](https://mlsanity.com/paper/2506.03169/) (2025)

## Abstract (arXiv)

Previous work has demonstrated that virtual accelerometry data, extracted from videos using cross-modality transfer approaches like IMUTube, is beneficial for training complex and effective human activity recognition (HAR) models. Systems like IMUTube were originally designed to cover activities that are based on substantial body (part) movements. Yet, life is complex, and a range of activities of daily living is based on only rather subtle movements, which bears the question to what extent systems like IMUTube are of value also for fine-grained HAR, i.e., When does IMUTube break? In this work we first introduce a measure to quantitatively assess the subtlety of human movements that are underlying activities of interest--the motion subtlety index (MSI)--which captures local pixel movements and pose changes in the vicinity of target virtual sensor locations, and correlate it to the eventual activity recognition accuracy. We then perform a "stress-test" on IMUTube and explore for which activities with underlying subtle movements a cross-modality transfer approach works, and for which not. As such, the work presented in this paper allows us to map out the landscape for IMUTube applications in practical scenarios.