# SPOTR: Spatio-temporal Pose Transformers for Human Motion Prediction

arXiv: https://arxiv.org/abs/2303.06277 · HTML: https://mlsanity.com/paper/2303.06277/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 30 references that are in the map.
- First posted on arXiv 11 Mar 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Attention Augmented Convolutional Networks](https://mlsanity.com/paper/1904.09925/) (2019)
- [Meshed-Memory Transformer for Image Captioning](https://mlsanity.com/paper/1912.08226/) (2019)
- [Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/1802.04687/) (2018)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation](https://mlsanity.com/paper/2003.13830/) (2020)
- [Learning Trajectory Dependencies for Human Motion Prediction](https://mlsanity.com/paper/1908.05436/) (2019)
- [Music Transformer](https://mlsanity.com/paper/1809.04281/) (2018)
- [History Repeats Itself: Human Motion Prediction via Motion Attention](https://mlsanity.com/paper/2007.11755/) (2020)
- [Character-Level Language Modeling with Deeper Self-Attention](https://mlsanity.com/paper/1808.04444/) (2018)
- [A Spatio-temporal Transformer for 3D Human Motion Prediction](https://mlsanity.com/paper/2004.08692/) (2020)
- [Convolutional Sequence to Sequence Model for Human Dynamics](https://mlsanity.com/paper/1805.00655/) (2018)
- [Human Motion Prediction via Spatio-Temporal Inpainting](https://mlsanity.com/paper/1812.05478/) (2018)
- [QuaterNet: A Quaternion-based Recurrent Model for Human Motion](https://mlsanity.com/paper/1805.06485/) (2018)
- [MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction](https://mlsanity.com/paper/2108.07152/) (2021)
- [Skeleton-based Action Recognition via Spatial and Temporal Transformer Networks](https://mlsanity.com/paper/2008.07404/) (2020)
- [Learning Human Motion Models for Long-term Predictions](https://mlsanity.com/paper/1704.02827/) (2017)
- [Pose Transformers (POTR): Human Motion Prediction with Non-Autoregressive Transformers](https://mlsanity.com/paper/2109.07531/) (2021)
- [Imitation Learning for Human Pose Prediction](https://mlsanity.com/paper/1909.03449/) (2019)
- [Multitask Non-Autoregressive Model for Human Motion Prediction](https://mlsanity.com/paper/2007.06426/) (2020)
- [Motion Transformer for Unsupervised Image Animation](https://mlsanity.com/paper/2209.14024/) (2022)
- [Multi Scale Temporal Graph Networks For Skeleton-based Action Recognition](https://mlsanity.com/paper/2012.02970/) (2020)
- [Application of Graph Convolutions in a Lightweight Model for Skeletal Human Motion Forecasting](https://mlsanity.com/paper/2110.04810/) (2021)

## Built on by (most cited first, 4 of 4)

- [Uncertainty-aware Probabilistic 3D Human Motion Forecasting via Invertible Networks](https://mlsanity.com/paper/2507.14694/) (2025)
- [A Mixture of Experts Approach to 3D Human Motion Prediction](https://mlsanity.com/paper/2405.06088/) (2024)
- [Long-Term Prediction of Local and Global Human Motion with Occlusion Recovery](https://mlsanity.com/paper/2606.27900/) (2026)
- [Gaussian-Mixture Latent Flow for Stochastic 3D Human Motion Prediction](https://mlsanity.com/paper/2608.21093/) (2026)

## Abstract (arXiv)

3D human motion prediction is a research area of high significance and a challenge in computer vision. It is useful for the design of many applications including robotics and autonomous driving. Traditionally, autogregressive models have been used to predict human motion. However, these models have high computation needs and error accumulation that make it difficult to use them for realtime applications. In this paper, we present a non-autogressive model for human motion prediction. We focus on learning spatio-temporal representations non-autoregressively for generation of plausible future motions. We propose a novel architecture that leverages the recently proposed Transformers. Human motion involves complex spatio-temporal dynamics with joints affecting the position and rotation of each other even though they are not connected directly. The proposed model extracts these dynamics using both convolutions and the self-attention mechanism. Using specialized spatial and temporal self-attention to augment the features extracted through convolution allows our model to generate spatio-temporally coherent predictions in parallel independent of the activity. Our contributions are threefold: (i) we frame human motion prediction as a sequence-to-sequence problem and propose a non-autoregressive Transformer to forecast a sequence of poses in parallel; (ii) our method is activity agnostic; (iii) we show that despite its simplicity, our approach is able to make accurate predictions, achieving better or comparable results compared to the state-of-the-art on two public datasets, with far fewer parameters and much faster inference.