# A Distributed Multi-Modal Sensing Approach for Human Activity Recognition in Real-Time Human-Robot Collaboration

arXiv: https://arxiv.org/abs/2602.07024 · HTML: https://mlsanity.com/paper/2602.07024/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 48 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Builds on (most cited first, 11 of 11)

- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis](https://mlsanity.com/paper/2011.06961/) (2020)
- [Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognition](https://mlsanity.com/paper/1711.11152/) (2017)
- [Object Handovers: a Review for Robotics](https://mlsanity.com/paper/2007.12952/) (2020)
- [A Multimodal Handover Failure Detection Dataset and Baselines](https://mlsanity.com/paper/2402.18319/) (2024)
- [Gaze-Based Intention Recognition for Human-Robot Collaboration](https://mlsanity.com/paper/2405.07570/) (2024)
- [A Multimodal Data Set of Human Handovers with Design Implications for Human-Robot Handovers](https://mlsanity.com/paper/2304.02154/) (2023)
- [A modular architecture for IMU-based data gloves](https://mlsanity.com/paper/2401.13254/) (2024)
- [A Systematic Review on Custom Data Gloves](https://mlsanity.com/paper/2405.15417/) (2024)
- [A Modular Framework for Flexible Planning in Human-Robot Collaboration](https://mlsanity.com/paper/2406.04907/) (2024)
- [A Comparative Study of Human Activity Recognition: Motion, Tactile, and multi-modal Approaches](https://mlsanity.com/paper/2505.08657/) (2025)

## Abstract (arXiv)

Human activity recognition (HAR) is fundamental in human-robot collaboration (HRC), enabling robots to respond to and dynamically adapt to human intentions. This paper introduces a HAR system combining a modular data glove equipped with Inertial Measurement Units and a vision-based tactile sensor to capture hand activities in contact with a robot. We tested our activity recognition approach under different conditions, including offline classification of segmented sequences, real-time classification under static conditions, and a realistic HRC scenario. The experimental results show a high accuracy for all the tasks, suggesting that multiple collaborative settings could benefit from this multi-modal approach.