mlsanityData to 7 Oct 2026

A Spatio-temporal Transformer for 3D Human Motion Prediction

arXiv 2004.08692 ↗First posted 18 Apr 2020cs.CVcited by 133

Not on the map. Why?

Abstract (arXiv)

We propose a novel Transformer-based architecture for the task of generative modelling of 3D human motion. Previous work commonly relies on RNN-based models considering shorter forecast horizons reaching a stationary and often implausible state quickly. Recent studies show that implicit temporal representations in the frequency domain are also effective in making predictions for a predetermined horizon. Our focus lies on learning spatio-temporal representations autoregressively and hence generation of plausible future developments over both short and long term. The proposed model learns high dimensional embeddings for skeletal joints and how to compose a temporally coherent pose via a decoupled temporal and spatial self-attention mechanism. Our dual attention concept allows the model to access current and past information directly and to capture both the structural and the temporal dependencies explicitly. We show empirically that this effectively learns the underlying motion dynamics and reduces error accumulation over time observed in auto-regressive models. Our model is able to make accurate short-term predictions and generate plausible motion sequences over long horizons. We make our code publicly available at https://github.com/eth-ait/motion-transformer.

Key facts. Cited by 133 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 28; 2023: 39; 2024: 26; 2025: 16; 2026: 16.

133 papers.

Sort:

By area:

  1. 2022Human Motion Diffusion Modelcited by 1,062
  2. 2021FaceFormer: Speech-Driven 3D Facial Animation with Transformerscited by 208
  3. 2022AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensingcited by 107
  4. 2021MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Predictioncited by 100
  5. 2022Back to MLP: A Simple Baseline for Human Motion Predictioncited by 99
  6. 2022BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Predictioncited by 82
  7. 2022GIMO: Gaze-Informed Human Motion Prediction in Contextcited by 81
  8. 2023CIRCLE: Capture In Rich Contextual Environmentscited by 79
  9. 2022Transformer Inertial Poser: Real-time Human Motion Reconstruction from Sparse IMUs with Simultaneous Terrain Generationcited by 75
  10. 2021Multi-Person Extreme Motion Predictioncited by 74
  11. 2022Progressively Generating Better Initial Guesses Towards Next Stages for High-Quality Human Motion Predictioncited by 66
  12. 2022NeMF: Neural Motion Fields for Kinematic Animationcited by 66
38 more
  1. 2023ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulationcited by 53
  2. 2021TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wildcited by 52
  3. 2023Physically Plausible Full-Body Hand-Object Interaction Synthesiscited by 52
  4. 2021Pose Transformers (POTR): Human Motion Prediction with Non-Autoregressive Transformerscited by 46
  5. 2022MotionMixer: MLP-based 3D Human Body Pose Forecastingcited by 42
  6. 2022Interaction Transformer for Human Reaction Generationcited by 42
  7. 2023Multimodal Motion Conditioned Diffusion Model for Skeleton-based Video Anomaly Detectioncited by 42
  8. 2023Realistic Full-Body Tracking from Sparse Observations via Joint-Level Modelingcited by 41
  9. 2023Generating Continual Human Motion in Diverse 3D Scenescited by 39
  10. 2023Motion Flow Matching for Human Motion Synthesis and Editingcited by 36
  11. 2022Global-local Motion Transformer for Unsupervised Skeleton-based Action Learningcited by 33
  12. 2022A generic diffusion-based approach for 3D human pose prediction in the wildcited by 33
  13. 2022Diverse Human Motion Prediction via Gumbel-Softmax Sampling from an Auxiliary Spacecited by 32
  14. 2021Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Predictioncited by 30
  15. 2022Motron: Multimodal Probabilistic Human Motion Forecastingcited by 30
  16. 2022Human Joint Kinematics Diffusion-Refinement for Stochastic Motion Predictioncited by 28
  17. 2023Interactive Humanoid: Online Full-Body Motion Reaction Synthesis with Social Affordance Canonicalization and Forecastingcited by 27
  18. 2023Plan, Posture and Go: Towards Open-World Text-to-Motion Generationcited by 26
  19. 2024WANDR: Intention-guided Human Motion Generationcited by 26
  20. 2022Conditional Motion In-betweeningcited by 25
  21. 2022MoCapAct: A Multi-Task Dataset for Simulated Humanoid Controlcited by 24
  22. 2022STNDT: Modeling Neural Population Activity with a Spatiotemporal Transformercited by 23
  23. 20223D Vision with Transformers: A Surveycited by 21
  24. 2023CoMusion: Towards Consistent Stochastic Human Motion Prediction via Motion Diffusioncited by 21
  25. 2023Auxiliary Tasks Benefit 3D Skeleton-based Human Motion Predictioncited by 20
  26. 2024PhysReaction: Physically Plausible Real-Time Humanoid Reaction Synthesis via Forward Dynamics Guided 4D Imitationcited by 18
  27. 2025UniEgoMotion: A Unified Model for Egocentric Motion Reconstruction, Forecasting, and Generationcited by 17
  28. 2023TransFusion: A Practical and Effective Transformer-based Diffusion Model for 3D Human Motion Predictioncited by 16
  29. 2024HOIMotion: Forecasting Human Motion During Human-Object Interactions Using Egocentric 3D Object Bounding Boxescited by 16
  30. 2021Tracking People by Predicting 3D Appearance, Location & Posecited by 15
  31. 2022H4D: Human 4D Modeling by Learning Neural Compositional Representationcited by 15
  32. 2022PoseBERT: A Generic Transformer Module for Temporal 3D Human Modelingcited by 15
  33. 2025Unified Uncertainty-Aware Diffusion for Multi-Agent Trajectory Modelingcited by 15
  34. 2023Can We Use Diffusion Probabilistic Models for 3D Motion Prediction?cited by 14
  35. 2024Learning Semantic Latent Directions for Accurate and Controllable Human Motion Predictioncited by 14
  36. 2025Nonisotropic Gaussian Diffusion for Realistic 3D Human Motion Predictioncited by 12
  37. 20223D Pose Based Feedback for Physical Exercisescited by 11
  38. 2023Guiding Online Reinforcement Learning with Action-Free Offline Pretrainingcited by 11

50 most cited of 133 All 133 (JSON)