# Pose Transformers (POTR): Human Motion Prediction with Non-Autoregressive Transformers

arXiv: https://arxiv.org/abs/2109.07531 · HTML: https://mlsanity.com/paper/2109.07531/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 15 Sep 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1711.02281/) (2017)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [Learning Trajectory Dependencies for Human Motion Prediction](https://mlsanity.com/paper/1908.05436/) (2019)
- [History Repeats Itself: Human Motion Prediction via Motion Attention](https://mlsanity.com/paper/2007.11755/) (2020)
- [A Spatio-temporal Transformer for 3D Human Motion Prediction](https://mlsanity.com/paper/2004.08692/) (2020)
- [Convolutional Sequence to Sequence Model for Human Dynamics](https://mlsanity.com/paper/1805.00655/) (2018)
- [Human Motion Prediction via Spatio-Temporal Inpainting](https://mlsanity.com/paper/1812.05478/) (2018)
- [2D/3D Pose Estimation and Action Recognition using Multitask Deep Learning](https://mlsanity.com/paper/1802.09232/) (2018)
- [Structured Prediction Helps 3D Human Motion Modelling](https://mlsanity.com/paper/1910.09070/) (2019)
- [Multitask Non-Autoregressive Model for Human Motion Prediction](https://mlsanity.com/paper/2007.06426/) (2020)

## Built on by (most cited first, 30 of 46)

- [BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Prediction](https://mlsanity.com/paper/2211.14304/) (2022)
- [GIMO: Gaze-Informed Human Motion Prediction in Context](https://mlsanity.com/paper/2204.09443/) (2022)
- [Interaction Transformer for Human Reaction Generation](https://mlsanity.com/paper/2207.01685/) (2022)
- [A generic diffusion-based approach for 3D human pose prediction in the wild](https://mlsanity.com/paper/2210.05669/) (2022)
- [Diverse Human Motion Prediction via Gumbel-Softmax Sampling from an Auxiliary Space](https://mlsanity.com/paper/2207.07351/) (2022)
- [3D Human Motion Prediction: A Survey](https://mlsanity.com/paper/2203.01593/) (2022)
- [Motion Question Answering via Modular Motion Programs](https://mlsanity.com/paper/2305.08953/) (2023)
- [Conditional Motion In-betweening](https://mlsanity.com/paper/2202.04307/) (2022)
- [SoMoFormer: Multi-Person Pose Forecasting with Transformers](https://mlsanity.com/paper/2208.14023/) (2022)
- [3D Vision with Transformers: A Survey](https://mlsanity.com/paper/2208.04309/) (2022)
- [TransFusion: A Practical and Effective Transformer-based Diffusion Model for 3D Human Motion Prediction](https://mlsanity.com/paper/2307.16106/) (2023)
- [UniEgoMotion: A Unified Model for Egocentric Motion Reconstruction, Forecasting, and Generation](https://mlsanity.com/paper/2508.01126/) (2025)
- [Learning Semantic Latent Directions for Accurate and Controllable Human Motion Prediction](https://mlsanity.com/paper/2407.11494/) (2024)
- [MotionAug: Augmentation with Physical Correction for Human Motion Prediction](https://mlsanity.com/paper/2203.09116/) (2022)
- [Nonisotropic Gaussian Diffusion for Realistic 3D Human Motion Prediction](https://mlsanity.com/paper/2501.06035/) (2025)
- [GaitForeMer: Self-Supervised Pre-Training of Transformers via Human Motion Forecasting for Few-Shot Gait Impairment Severity Estimation](https://mlsanity.com/paper/2207.00106/) (2022)
- [GazeMoDiff: Gaze-guided Diffusion Model for Stochastic Human Motion Prediction](https://mlsanity.com/paper/2312.12090/) (2023)
- [Dyadic Human Motion Prediction](https://mlsanity.com/paper/2112.00396/) (2021)
- [PreTR: Spatio-Temporal Non-Autoregressive Trajectory Prediction Transformer](https://mlsanity.com/paper/2203.09293/) (2022)
- [Multimodal Sense-Informed Prediction of 3D Human Motions](https://mlsanity.com/paper/2405.02911/) (2024)
- [Benchmarking Skeleton-based Motion Encoder Models for Clinical Applications: Estimating Parkinson's Disease Severity in Walking Sequences](https://mlsanity.com/paper/2405.17817/) (2024)
- [CARE-PD: A Multi-Site Anonymized Clinical Dataset for Parkinson's Disease Gait Assessment](https://mlsanity.com/paper/2510.04312/) (2025)
- [Learning Disentangled Representations for Controllable Human Motion Prediction](https://mlsanity.com/paper/2207.01388/) (2022)
- [FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observations](https://mlsanity.com/paper/2211.14309/) (2022)
- [Mask prior-guided denoising diffusion improves inverse protein folding](https://mlsanity.com/paper/2412.07815/) (2024)
- [DMMGAN: Diverse Multi Motion Prediction of 3D Human Joints using Attention-Based Generative Adverserial Network](https://mlsanity.com/paper/2209.09124/) (2022)
- [Multi-Scale Control Signal-Aware Transformer for Motion Synthesis without Phase](https://mlsanity.com/paper/2303.01685/) (2023)
- [SPOTR: Spatio-temporal Pose Transformers for Human Motion Prediction](https://mlsanity.com/paper/2303.06277/) (2023)
- [AnyPose: Anytime 3D Human Pose Forecasting via Neural Ordinary Differential Equations](https://mlsanity.com/paper/2309.04840/) (2023)
- [CacheFlow: Fast Human Motion Prediction by Cached Normalizing Flow](https://mlsanity.com/paper/2505.13140/) (2025)

## Abstract (arXiv)

We propose to leverage Transformer architectures for non-autoregressive human motion prediction. Our approach decodes elements in parallel from a query sequence, instead of conditioning on previous predictions such as instate-of-the-art RNN-based approaches. In such a way our approach is less computational intensive and potentially avoids error accumulation to long term elements in the sequence. In that context, our contributions are fourfold: (i) we frame human motion prediction as a sequence-to-sequence problem and propose a non-autoregressive Transformer to infer the sequences of poses in parallel; (ii) we propose to decode sequences of 3D poses from a query sequence generated in advance with elements from the input sequence;(iii) we propose to perform skeleton-based activity classification from the encoder memory, in the hope that identifying the activity can improve predictions;(iv) we show that despite its simplicity, our approach achieves competitive results in two public datasets, although surprisingly more for short term predictions rather than for long term ones.