# TH"OR-MAGNI Act: Actions for Human Motion Modeling in Robot-Shared Industrial Spaces

arXiv: https://arxiv.org/abs/2412.13729 · HTML: https://mlsanity.com/paper/2412.13729/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 19 references that are in the map.
- First posted on arXiv 18 Dec 2024.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Human Motion Trajectory Prediction: A Survey](https://mlsanity.com/paper/1905.06113/) (2019)
- [Long-term Human Motion Prediction with Scene Context](https://mlsanity.com/paper/2007.03672/) (2020)
- [Human Trajectory Forecasting in Crowds: A Deep Learning Perspective](https://mlsanity.com/paper/2007.03639/) (2020)
- [Core Challenges of Social Robot Navigation: A Survey](https://mlsanity.com/paper/2103.05668/) (2021)
- [TITAN: Future Forecast using Action Priors](https://mlsanity.com/paper/2003.13886/) (2020)
- [LOKI: Long Term and Key Intentions for Trajectory Prediction](https://mlsanity.com/paper/2108.08236/) (2021)
- [JRDB-Act: A Large-scale Dataset for Spatio-temporal Action, Social Group and Activity Detection](https://mlsanity.com/paper/2106.08827/) (2021)
- [Long-Term Human Trajectory Prediction using 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2405.00552/) (2024)
- [The Magni Human Motion Dataset: Accurate, Complex, Multi-Modal, Natural, Semantically-Rich and Contextualized](https://mlsanity.com/paper/2208.14925/) (2022)
- [CLiFF-LHMP: Using Spatial Dynamics Patterns for Long-Term Human Motion Prediction](https://mlsanity.com/paper/2309.07066/) (2023)
- [Human Gaze and Head Rotation during Navigation, Exploration and Object Manipulation in Shared Environments with Robots](https://mlsanity.com/paper/2406.06300/) (2024)

## Built on by (most cited first, 1 of 1)

- [Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Review](https://mlsanity.com/paper/2506.14831/) (2025)

## Abstract (arXiv)

Accurate human activity and trajectory prediction are crucial for ensuring safe and reliable human-robot interactions in dynamic environments, such as industrial settings, with mobile robots. Datasets with fine-grained action labels for moving people in industrial environments with mobile robots are scarce, as most existing datasets focus on social navigation in public spaces. This paper introduces the TH"OR-MAGNI Act dataset, a substantial extension of the TH"OR-MAGNI dataset, which captures participant movements alongside robots in diverse semantic and spatial contexts. TH"OR-MAGNI Act provides 8.3 hours of manually labeled participant actions derived from egocentric videos recorded via eye-tracking glasses. These actions, aligned with the provided TH"OR-MAGNI motion cues, follow a long-tailed distribution with diversified acceleration, velocity, and navigation distance profiles. We demonstrate the utility of TH"OR-MAGNI Act for two tasks: action-conditioned trajectory prediction and joint action and trajectory prediction. We propose two efficient transformer-based models that outperform the baselines to address these tasks. These results underscore the potential of TH"OR-MAGNI Act to develop predictive models for enhanced human-robot interaction in complex environments.