# Attention-Oriented Action Recognition for Real-Time Human-Robot Interaction

arXiv: https://arxiv.org/abs/2007.01065 · HTML: https://mlsanity.com/paper/2007.01065/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2020.

## Built on by (most cited first, 3 of 3)

- [HHP-Net: A light Heteroscedastic neural network for Head Pose estimation with uncertainty](https://mlsanity.com/paper/2111.01440/) (2021)
- [Towards Ubiquitous Semantic Metaverse: Challenges, Approaches, and Opportunities](https://mlsanity.com/paper/2307.06687/) (2023)
- [Exploring Self-supervised Skeleton-based Action Recognition in Occluded Environments](https://mlsanity.com/paper/2309.12029/) (2023)

## Abstract (arXiv)

Despite the notable progress made in action recognition tasks, not much work has been done in action recognition specifically for human-robot interaction. In this paper, we deeply explore the characteristics of the action recognition task in interaction scenarios and propose an attention-oriented multi-level network framework to meet the need for real-time interaction. Specifically, a Pre-Attention network is employed to roughly focus on the interactor in the scene at low resolution firstly and then perform fine-grained pose estimation at high resolution. The other compact CNN receives the extracted skeleton sequence as input for action recognition, utilizing attention-like mechanisms to capture local spatial-temporal patterns and global semantic information effectively. To evaluate our approach, we construct a new action dataset specially for the recognition task in interaction scenarios. Experimental results on our dataset and high efficiency (112 fps at 640 x 480 RGBD) on the mobile computing platform (Nvidia Jetson AGX Xavier) demonstrate excellent applicability of our method on action recognition in real-time human-robot interaction.