mlsanityData to 7 Oct 2026

Multi-Person 3D Motion Prediction with Multi-Range Transformers

arXiv 2111.12073 ↗First posted 23 Nov 2021cs.CVcited by 63

Not on the map. Why?

Abstract (arXiv)

We propose a novel framework for multi-person 3D motion trajectory prediction. Our key observation is that a human's action and behaviors may highly depend on the other persons around. Thus, instead of predicting each human pose trajectory in isolation, we introduce a Multi-Range Transformers model which contains of a local-range encoder for individual motion and a global-range encoder for social interactions. The Transformer decoder then performs prediction for each person by taking a corresponding pose as a query which attends to both local and global-range encoder features. Our model not only outperforms state-of-the-art methods on long-term 3D motion prediction, but also generates diverse social interactions. More interestingly, our model can even predict 15-person motion simultaneously by automatically dividing the persons into different interaction groups. Project page with code is available at https://jiashunwang.github.io/MRT/.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 51 of its 76 references that are in the map. First posted on arXiv 23 Nov 2021.

Builds on

51 of 76 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  7. 2017Non-local Neural Networkscited by 2,290
  8. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  9. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  10. 2018Image Transformercited by 770
  11. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  12. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
38 more
  1. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  2. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 497
  3. 2017On human motion prediction using recurrent neural networkscited by 336
  4. 2020Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Predictioncited by 324
  5. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  6. 2017Relation Networks for Object Detectioncited by 305
  7. 2018SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraintscited by 296
  8. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 245
  9. 2020It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Predictioncited by 235
  10. 2015Structural-RNN: Deep Learning on Spatio-Temporal Graphscited by 234
  11. 2019Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networkscited by 228
  12. 2020Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Predictioncited by 218
  13. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  14. 2020Long-term Human Motion Prediction with Scene Contextcited by 191
  15. 2020History Repeats Itself: Human Motion Prediction via Motion Attentioncited by 164
  16. 2020Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenescited by 135
  17. 2019Generating 3D People in Scenes without Peoplecited by 127
  18. 2018Convolutional Sequence to Sequence Model for Human Dynamicscited by 115
  19. 2020We are More than Our Joints: Predicting how 3D Bodies Movecited by 115
  20. 2018Human Motion Prediction via Spatio-Temporal Inpaintingcited by 114
  21. 2021SimPoE: Simulated Character Control for 3D Human Pose Estimationcited by 111
  22. 2018QuaterNet: A Quaternion-based Recurrent Model for Human Motioncited by 108
  23. 2021Pose Recognition with Cascade Transformerscited by 99
  24. 2017Deep representation learning for human motion prediction and classificationcited by 97
  25. 2018A Neural Temporal Model for Human Motion Predictioncited by 87
  26. 2020Learning to Generate Diverse Dance Motions with Transformercited by 85
  27. 2019Diverse Trajectory Forecasting with Determinantal Point Processescited by 80
  28. 2019Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories with GANscited by 78
  29. 2017Learning Human Motion Models for Long-term Predictionscited by 67
  30. 2020Recursive Social Behavior Graph for Trajectory Predictioncited by 66
  31. 2021TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wildcited by 52
  32. 2019Predicting 3D Human Dynamics from Videocited by 51
  33. 2020Socially and Contextually Aware Human Motion and Pose Forecastingcited by 51
  34. 2021Three Steps to Multimodal Trajectory Prediction: Modality Clustering, Classification and Synthesiscited by 46
  35. 2017Soft + Hardwired Attention: An LSTM Framework for Human Trajectory Prediction and Abnormal Event Detectioncited by 42
  36. 2021TFPose: Direct Human Pose Estimation with Transformerscited by 42
  37. 2021End-to-End Trainable Multi-Instance Pose Estimation with Transformerscited by 30
  38. 2016Learning Social Affordance for Human-Robot Interactioncited by 20

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 24; 2024: 12; 2025: 12; 2026: 6.

63 papers.

Sort:

By area:

  1. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 406
  2. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
  3. 2023Human Motion Diffusion as a Generative Priorcited by 354
  4. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 267
  5. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
  6. 2023ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactionscited by 62
  7. 2023Music-Driven Group Choreographycited by 36
  8. 2023Uncertainty-aware State Space Transformer for Egocentric 3D Hand Trajectory Forecastingcited by 33
  9. 2024Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioningcited by 29
  10. 2023Stochastic Multi-Person 3D Motion Forecastingcited by 28
  11. 2024Towards Open Domain Text-Driven Synthesis of Multi-Person Motionscited by 27
  12. 2023Trajectory-Aware Body Interaction Transformer for Multi-Person Pose Forecastingcited by 25
38 more
  1. 2025A Survey on Human Interaction Motion Generationcited by 25
  2. 2022SoMoFormer: Multi-Person Pose Forecasting with Transformerscited by 23
  3. 2023InterControl: Zero-shot Human Interaction Generation by Controlling Every Jointcited by 22
  4. 2023Diverse 3D Hand Gesture Prediction from Body Dynamics by Bilateral Hand Disentanglementcited by 18
  5. 2023Controllable Group Choreography using Contrastive Diffusioncited by 17
  6. 2023ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimationcited by 16
  7. 2023TransFusion: A Practical and Effective Transformer-based Diffusion Model for 3D Human Motion Predictioncited by 16
  8. 2023Joint-Relation Transformer for Multi-Person Motion Predictioncited by 16
  9. 2024Frequency Guidance Matters: Skeletal Action Recognition by Frequency-Aware Mixed Transformercited by 15
  10. 2022Didn't see that coming: a survey on non-verbal social human behavior forecastingcited by 12
  11. 2023InteRACT: Transformer Models for Human Intent Prediction Conditioned on Robot Actionscited by 10
  12. 2022Comparison of Spatio-Temporal Models for Human Motion and Pose Forecasting in Face-to-Face Interaction Scenarioscited by 8
  13. 2023Best Practices for 2-Body Pose Forecastingcited by 8
  14. 2023Weakly Supervised Multi-Modal 3D Human Body Pose Estimation for Autonomous Drivingcited by 8
  15. 2023Social Motion Prediction with Cognitive Hierarchiescited by 8
  16. 2025Learning Bimanual Manipulation via Action Chunking and Inter-Arm Coordination with Transformerscited by 8
  17. 2024Robot Interaction Behavior Generation based on Social Motion Forecasting for Human-Robot Interactioncited by 7
  18. 2025Ponimator: Unfolding Interactive Pose for Versatile Human-human Interaction Animationcited by 7
  19. 2022I^2R-Net: Intra- and Inter-Human Relation Network for Multi-Person Pose Estimationcited by 6
  20. 2022SoMoFormer: Social-Aware Motion Transformer for Multi-Person Motion Predictioncited by 6
  21. 2023Staged Contact-Aware Global Human Motion Forecastingcited by 6
  22. 2024Multi-Condition Latent Diffusion Network for Scene-Aware Neural Human Motion Predictioncited by 6
  23. 2025Frequency-Semantic Enhanced Variational Autoencoder for Zero-Shot Skeleton-based Action Recognitioncited by 6
  24. 2022Human-Robot Commensality: Bite Timing Prediction for Robot-Assisted Feeding in Groupscited by 5
  25. 2023Expressive Forecasting of 3D Whole-body Human Motionscited by 5
  26. 2024Human Motion Prediction under Unexpected Perturbationcited by 5
  27. 2025Waymo-3DSkelMo: A Multi-Agent 3D Skeletal Motion Dataset for Pedestrian Interaction Modeling in Autonomous Drivingcited by 5
  28. 2023Multi-Scale Control Signal-Aware Transformer for Motion Synthesis without Phasecited by 4
  29. 2023ManiCast: Collaborative Manipulation with Cost-Aware Human Forecastingcited by 4
  30. 2025Dyadic Mamba: Long-term Dyadic Human Motion Synthesiscited by 4
  31. 2025Efficient Multi-Person Motion Prediction by Lightweight Spatial and Temporal Interactionscited by 4
  32. 2023The MI-Motion Dataset and Benchmark for 3D Multi-Person Motion Predictioncited by 3
  33. 2026Modeling 3D Pedestrian-Vehicle Interactions for Vehicle-Conditioned Pose Forecastingcited by 3
  34. 2025HUMOF: Human Motion Forecasting in Interactive Social Scenescited by 2
  35. 2025MoReact: Generating Reactive Motion from Textual Descriptionscited by 2
  36. 2023PGformer: Proxy-Bridged Game Transformer for Multi-Person Highly Interactive Extreme Motion Predictioncited by 1
  37. 2024Graph and Skipped Transformer: Exploiting Spatial and Temporal Modeling Capacities for Efficient 3D Human Pose Estimationcited by 1
  38. 2024MDMP: Multi-modal Diffusion for supervised Motion Predictions with uncertaintycited by 1

50 most cited of 63 All 63 (JSON)