# A Comparative Study of Human Activity Recognition: Motion, Tactile, and multi-modal Approaches

arXiv: https://arxiv.org/abs/2505.08657 · HTML: https://mlsanity.com/paper/2505.08657/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 43 references that are in the map.
- First posted on arXiv 13 May 2025.

## Builds on (most cited first, 9 of 9)

- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Human Action Recognition from Various Data Modalities: A Review](https://mlsanity.com/paper/2012.11866/) (2020)
- [Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis](https://mlsanity.com/paper/2011.06961/) (2020)
- [Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognition](https://mlsanity.com/paper/1711.11152/) (2017)
- [A Multimodal Handover Failure Detection Dataset and Baselines](https://mlsanity.com/paper/2402.18319/) (2024)
- [Gaze-Based Intention Recognition for Human-Robot Collaboration](https://mlsanity.com/paper/2405.07570/) (2024)
- [A Multimodal Data Set of Human Handovers with Design Implications for Human-Robot Handovers](https://mlsanity.com/paper/2304.02154/) (2023)
- [A modular architecture for IMU-based data gloves](https://mlsanity.com/paper/2401.13254/) (2024)
- [A Modular Framework for Flexible Planning in Human-Robot Collaboration](https://mlsanity.com/paper/2406.04907/) (2024)

## Built on by (most cited first, 1 of 1)

- [A Distributed Multi-Modal Sensing Approach for Human Activity Recognition in Real-Time Human-Robot Collaboration](https://mlsanity.com/paper/2602.07024/) (2026)

## Abstract (arXiv)

Human activity recognition (HAR) is essential for effective Human-Robot Collaboration (HRC), enabling robots to interpret and respond to human actions. This study evaluates the ability of a vision-based tactile sensor to classify 15 activities, comparing its performance to an IMU-based data glove. Additionally, we propose a multi-modal framework combining tactile and motion data to leverage their complementary strengths. We examined three approaches: motion-based classification (MBC) using IMU data, tactile-based classification (TBC) with single or dual video streams, and multi-modal classification (MMC) integrating both. Offline validation on segmented datasets assessed each configuration's accuracy under controlled conditions, while online validation on continuous action sequences tested online performance. Results showed the multi-modal approach consistently outperformed single-modality methods, highlighting the potential of integrating tactile and motion sensing to enhance HAR systems for collaborative robotics.