# STPOTR: Simultaneous Human Trajectory and Pose Prediction Using a Non-Autoregressive Transformer for Robot Following Ahead

arXiv: https://arxiv.org/abs/2209.07600 · HTML: https://mlsanity.com/paper/2209.07600/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 15 Sep 2022.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Learning Trajectory Dependencies for Human Motion Prediction](https://mlsanity.com/paper/1908.05436/) (2019)
- [DLow: Diversifying Latent Flows for Diverse Human Motion Prediction](https://mlsanity.com/paper/2003.08386/) (2020)
- [Transformer Networks for Trajectory Forecasting](https://mlsanity.com/paper/2003.08111/) (2020)
- [A Spatio-temporal Transformer for 3D Human Motion Prediction](https://mlsanity.com/paper/2004.08692/) (2020)
- [Generalization in Generation: A closer look at Exposure Bias](https://mlsanity.com/paper/1910.00292/) (2019)
- [Pose Transformers (POTR): Human Motion Prediction with Non-Autoregressive Transformers](https://mlsanity.com/paper/2109.07531/) (2021)
- [Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Prediction](https://mlsanity.com/paper/2108.11244/) (2021)
- [3D Human Motion Prediction: A Survey](https://mlsanity.com/paper/2203.01593/) (2022)
- [PreTR: Spatio-Temporal Non-Autoregressive Trajectory Prediction Transformer](https://mlsanity.com/paper/2203.09293/) (2022)
- [LBGP: Learning Based Goal Planning for Autonomous Following in Front](https://mlsanity.com/paper/2011.03125/) (2020)
- [Transformer based trajectory prediction](https://mlsanity.com/paper/2112.04350/) (2021)
- [Real-time 3D Human Tracking for Mobile Robots with Multisensors](https://mlsanity.com/paper/1703.04877/) (2017)

## Abstract (arXiv)

In this paper, we develop a neural network model to predict future human motion from an observed human motion history. We propose a non-autoregressive transformer architecture to leverage its parallel nature for easier training and fast, accurate predictions at test time. The proposed architecture divides human motion prediction into two parts: 1) the human trajectory, which is the hip joint 3D position over time and 2) the human pose which is the all other joints 3D positions over time with respect to a fixed hip joint. We propose to make the two predictions simultaneously, as the shared representation can improve the model performance. Therefore, the model consists of two sets of encoders and decoders. First, a multi-head attention module applied to encoder outputs improves human trajectory. Second, another multi-head self-attention module applied to encoder outputs concatenated with decoder outputs facilitates learning of temporal dependencies. Our model is well-suited for robotic applications in terms of test accuracy and speed, and compares favorably with respect to state-of-the-art methods. We demonstrate the real-world applicability of our work via the Robot Follow-Ahead task, a challenging yet practical case study for our proposed model.