mlsanityData to 7 Oct 2026

Joint-Relation Transformer for Multi-Person Motion Prediction

arXiv 2308.04808 ↗First posted 9 Aug 2023cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

Multi-person motion prediction is a challenging problem due to the dependency of motion on both individual past movements and interactions with other people. Transformer-based methods have shown promising results on this task, but they miss the explicit relation representation between joints, such as skeleton structure and pairwise distance, which is crucial for accurate interaction modeling. In this paper, we propose the Joint-Relation Transformer, which utilizes relation information to enhance interaction modeling and improve future motion prediction. Our relation information contains the relative distance and the intra-/inter-person physical constraints. To fuse relation and joint information, we design a novel joint-relation fusion layer with relation-aware attention to update both features. Additionally, we supervise the relation information by forecasting future distance. Experiments show that our method achieves a 13.4% improvement of 900ms VIM on 3DPW-SoMoF/RC and 17.8%/12.0% improvement of 3s MPJPE on CMU-Mpcap/MuPoTS-3D dataset.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 39 of its 58 references that are in the map. First posted on arXiv 9 Aug 2023.

Builds on

39 of 58 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  3. 2020A Generalization of Transformer Networks to Graphscited by 653
  4. 2021Rethinking Graph Transformers with Spectral Attentioncited by 468
  5. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  6. 2021Mesh Graphormercited by 264
  7. 2020The Open Catalyst 2020 (OC20) Dataset and Community Challengescited by 246
  8. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 245
  9. 2015Structural-RNN: Deep Learning on Spatio-Temporal Graphscited by 234
  10. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  11. 2022Representing Long-Range Context for Graph Neural Networks with Global Attentioncited by 193
  12. 2020Graph-Bert: Only Attention is Needed for Learning Graph Representationscited by 176
27 more
  1. 2020History Repeats Itself: Human Motion Prediction via Motion Attentioncited by 164
  2. 2018Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learningcited by 162
  3. 2021GraphiT: Encoding Graph Structure in Transformerscited by 146
  4. 2020Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Predictioncited by 132
  5. 2022GroupNet: Multiscale Hypergraph Neural Networks for Trajectory Prediction with Relational Reasoningcited by 128
  6. 2018Convolutional Sequence to Sequence Model for Human Dynamicscited by 115
  7. 2018QuaterNet: A Quaternion-based Recurrent Model for Human Motioncited by 108
  8. 2017Deep representation learning for human motion prediction and classificationcited by 97
  9. 2023EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoningcited by 97
  10. 2018A Neural Temporal Model for Human Motion Predictioncited by 87
  11. 2022Remember Intentions: Retrospective-Memory-based Trajectory Predictioncited by 86
  12. 2021Multi-Person Extreme Motion Predictioncited by 74
  13. 2017Learning Human Motion Models for Long-term Predictionscited by 67
  14. 2021Gophormer: Ego-Graph Transformer for Node Classificationcited by 63
  15. 2021Multi-Person 3D Motion Prediction with Multi-Range Transformerscited by 63
  16. 2018Action-Agnostic Human Pose Forecastingcited by 56
  17. 2021TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wildcited by 52
  18. 2020Socially and Contextually Aware Human Motion and Pose Forecastingcited by 51
  19. 2019Symbiotic Graph Neural Networks for 3D Skeleton-based Human Action Recognition and Motion Predictioncited by 50
  20. 2020Interpretable Rumor Detection in Microblogs by Attending to User Interactionscited by 44
  21. 2022Relational Attention: Generalizing Transformers for Graph-Structured Taskscited by 41
  22. 2021Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Predictioncited by 36
  23. 2023TBP-Former: Learning Temporal Bird's-Eye-View Pyramid for Joint Perception and Prediction in Vision-Centric Autonomous Drivingcited by 30
  24. 2023Stochastic Multi-Person 3D Motion Forecastingcited by 28
  25. 2022SoMoFormer: Multi-Person Pose Forecasting with Transformerscited by 23
  26. 2021Simple Baseline for Single Human Motion Forecastingcited by 18
  27. 2022Collaborative Uncertainty Benefits Multi-Agent Multi-Modal Trajectory Forecastingcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 6; 2026: 3.

16 papers.

Sort:

By area:

  1. 2024Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioningcited by 29
  2. 2024Language-Driven Interactive Traffic Trajectory Generationcited by 19
  3. 2024Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaptioncited by 18
  4. 2025Ponimator: Unfolding Interactive Pose for Versatile Human-human Interaction Animationcited by 7
  5. 2024Human Motion Prediction under Unexpected Perturbationcited by 5
  6. 2023Orientation-Aware Leg Movement Learning for Action-Driven Human Motion Predictioncited by 4
  7. 2025Efficient Multi-Person Motion Prediction by Lightweight Spatial and Temporal Interactionscited by 4
  8. 2025Large-Scale Multi-Character Interaction Synthesiscited by 3
  9. 2025HUMOF: Human Motion Forecasting in Interactive Social Scenescited by 2
  10. 2024Relation Learning and Aggregate-attention for Multi-person Motion Predictioncited by 0
  11. 2024UnityGraph: Unified Learning of Spatio-temporal features for Multi-person Motion Predictioncited by 0
  12. 2025Scriboora: Rethinking Human Pose Forecastingcited by 0
4 more
  1. 2025Spatiotemporal-Untrammelled Mixture of Experts for Multi-Person Motion Predictioncited by 0
  2. 2026Ego-Human Motion Prediction with 3D-Aware LLMcited by 0
  3. 2026Residual Flow Matching with Dynamic Cross-Interaction for 3D Multi-Person Motion Predictioncited by 0
  4. 2026Multi-Person Human Motion Forecasting in Complex Scenescited by 0