# Online Human Action Detection during Escorting

arXiv: https://arxiv.org/abs/2506.23573 · HTML: https://mlsanity.com/paper/2506.23573/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 48 references that are in the map.
- First posted on arXiv 30 Jun 2025.

## Builds on (most cited first, 23 of 23)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels](https://mlsanity.com/paper/2211.13977/) (2022)
- [You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection](https://mlsanity.com/paper/2106.00666/) (2021)
- [Playing for Benchmarks](https://mlsanity.com/paper/1709.07322/) (2017)
- [Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities](https://mlsanity.com/paper/2001.07416/) (2020)
- [Hierarchical Clustering with Hard-batch Triplet Loss for Person Re-identification](https://mlsanity.com/paper/1910.12278/) (2019)
- [Appearance-Preserving 3D Convolution for Video-based Person Re-identification](https://mlsanity.com/paper/2007.08434/) (2020)
- [Diversity Regularized Spatiotemporal Attention for Video-based Person Re-identification](https://mlsanity.com/paper/1803.09882/) (2018)
- [Home Action Genome: Cooperative Compositional Action Understanding](https://mlsanity.com/paper/2105.05226/) (2021)
- [Unsupervised Person Re-identification by Deep Learning Tracklet Association](https://mlsanity.com/paper/1809.02874/) (2018)
- [Video-based Person Re-identification with Spatial and Temporal Memory Networks](https://mlsanity.com/paper/2108.09039/) (2021)
- [Online Human Action Detection using Joint Classification-Regression Recurrent Neural Networks](https://mlsanity.com/paper/1604.05633/) (2016)
- [GateHUB: Gated History Unit with Background Suppression for Online Action Detection](https://mlsanity.com/paper/2206.04668/) (2022)
- [Human Activity Recognition Using Tools of Convolutional Neural Networks: A State of the Art Review, Data Sets, Challenges and Future Prospects](https://mlsanity.com/paper/2202.03274/) (2022)
- [Top-push Video-based Person Re-identification](https://mlsanity.com/paper/1604.08683/) (2016)
- [Enabling human-like task identification from natural conversation](https://mlsanity.com/paper/2008.10073/) (2020)
- [Can Visual Context Improve Automatic Speech Recognition for an Embodied Agent?](https://mlsanity.com/paper/2210.13189/) (2022)
- [tagE: Enabling an Embodied Agent to Understand Human Instructions](https://mlsanity.com/paper/2310.15605/) (2023)
- [How Much is too Much: Exploring the Effect of Verbal Route Description Length on Indoor Navigation](https://mlsanity.com/paper/2408.15367/) (2024)

## Abstract (arXiv)

The deployment of robot assistants in large indoor spaces has seen significant growth, with escorting tasks becoming a key application. However, most current escorting robots primarily rely on navigation-focused strategies, assuming that the person being escorted will follow without issue. In crowded environments, this assumption often falls short, as individuals may struggle to keep pace, become obstructed, get distracted, or need to stop unexpectedly. As a result, conventional robotic systems are often unable to provide effective escorting services due to their limited understanding of human movement dynamics. To address these challenges, an effective escorting robot must continuously detect and interpret human actions during the escorting process and adjust its movement accordingly. However, there is currently no existing dataset designed specifically for human action detection in the context of escorting. Given that escorting often occurs in crowded environments, where other individuals may enter the robot's camera view, the robot also needs to identify the specific human it is escorting (the subject) before predicting their actions. Since no existing model performs both person re-identification and action prediction in real-time, we propose a novel neural network architecture that can accomplish both tasks. This enables the robot to adjust its speed dynamically based on the escortee's movements and seamlessly resume escorting after any disruption. In comparative evaluations against strong baselines, our system demonstrates superior efficiency and effectiveness, showcasing its potential to significantly improve robotic escorting services in complex, real-world scenarios.