mlsanityData to 7 Oct 2026

Social Motion Prediction with Cognitive Hierarchies

arXiv 2311.04726 ↗First posted 8 Nov 2023cs.CVcited by 8

On the map: Offline RL and imitation learning from data → Social robot navigation

Abstract (arXiv)

Humans exhibit a remarkable capacity for anticipating the actions of others and planning their own actions accordingly. In this study, we strive to replicate this ability by addressing the social motion prediction problem. We introduce a new benchmark, a novel formulation, and a cognition-inspired framework. We present Wusi, a 3D multi-person motion dataset under the context of team sports, which features intense and strategic human interactions and diverse pose distributions. By reformulating the problem from a multi-agent reinforcement learning perspective, we incorporate behavioral cloning and generative adversarial imitation learning to boost learning efficiency and generalization. Furthermore, we take into account the cognitive aspects of the human social action planning process and develop a cognitive hierarchy framework to predict strategic human social interactions. We conduct comprehensive experiments to validate the effectiveness of our proposed dataset and approach. Code and data are available at https://walter0807.github.io/Social-CH/.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Social robot navigation, one of 13 directions in Offline RL and imitation learning from data. Builds on 41 of its 78 references that are in the map. First posted on arXiv 8 Nov 2023.

Builds on

41 of 78 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  7. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  8. 2020Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Predictioncited by 324
  9. 2019Towards Real-Time Multi-Object Trackingcited by 311
  10. 2018SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraintscited by 296
  11. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  12. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 245
29 more
  1. 2020It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Predictioncited by 235
  2. 2019Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networkscited by 228
  3. 2018Convolutional Social Pooling for Vehicle Trajectory Predictioncited by 221
  4. 2020Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Predictioncited by 218
  5. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  6. 2017PoseTrack: A Benchmark for Human Pose Estimation and Trackingcited by 173
  7. 2020History Repeats Itself: Human Motion Prediction via Motion Attentioncited by 164
  8. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
  9. 2023Human Motion Generation: A Surveycited by 95
  10. 2023HumanMAC: Masked Motion Completion for Human Motion Predictioncited by 81
  11. 2019Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories with GANscited by 78
  12. 2021Multi-Person Extreme Motion Predictioncited by 74
  13. 2019GRIP++: Enhanced Graph-based Interaction-aware Trajectory Prediction for Autonomous Drivingcited by 73
  14. 2020Recursive Social Behavior Graph for Trajectory Predictioncited by 66
  15. 2022SocialVAE: Human Trajectory Prediction using Timewise Latentscited by 65
  16. 2021Multi-Person 3D Motion Prediction with Multi-Range Transformerscited by 63
  17. 2021TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wildcited by 52
  18. 2020Socially and Contextually Aware Human Motion and Pose Forecastingcited by 51
  19. 2019Towards Social Artificial Intelligence: Nonverbal Social Signal Prediction in A Triadic Interactioncited by 48
  20. 2021Three Steps to Multimodal Trajectory Prediction: Modality Clustering, Classification and Synthesiscited by 46
  21. 2016PoseTrack: Joint Multi-Person Pose Estimation and Trackingcited by 44
  22. 2019Imitation Learning for Human Pose Predictioncited by 44
  23. 2022Social-Implicit: Rethinking Trajectory Prediction Evaluation and The Effectiveness of Implicit Maximum Likelihood Estimationcited by 40
  24. 2022Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic Projectioncited by 39
  25. 2018PORCA: Modeling and Planning for Autonomous Driving among Many Pedestrianscited by 36
  26. 2023Stochastic Multi-Person 3D Motion Forecastingcited by 28
  27. 2022SoMoFormer: Multi-Person Pose Forecasting with Transformerscited by 23
  28. 2021Simple Baseline for Single Human Motion Forecastingcited by 18
  29. 2022SoMoFormer: Social-Aware Motion Transformer for Multi-Person Motion Predictioncited by 6

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 4.

8 papers.

  1. 2024Language Models Represent Beliefs of Self and Otherscited by 29
  2. 2024in2IN: Leveraging individual Information to Generate Human INteractionscited by 29
  3. 2025Ready-to-React: Online Reaction Policy for Two-Character Interaction Generationcited by 25
  4. 2024SAT-HMR: Real-Time Multi-Person 3D Mesh Estimation via Scale-Adaptive Tokenscited by 15
  5. 2024CrowdMoGen: Zero-Shot Text-Driven Collective Motion Generationcited by 4
  6. 2025MoReact: Generating Reactive Motion from Textual Descriptionscited by 2
  7. 2025Learning Velocity and Acceleration: Self-Supervised Motion Consistency for Pedestrian Trajectory Predictioncited by 1
  8. 20253D Human-Human Interaction Anomaly Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Learning Group Interactions and Semantic Intentions for Multi-Object Trajectory Prediction
  2. 2020Social NCE: Contrastive Learning of Socially-aware Motion Representations
  3. 2026Trajectory Learning with Graph Representations for Social Robot Navigation
  4. 2025Diffusion-Based Imitation Learning for Social Pose Generation
  5. 2018Understanding Human Behaviors in Crowds by Imitating the Decision-Making Process
  6. 2020How Do We Move: Modeling Human Movement with System Dynamics ↗
  7. 2026Multi-Person Human Motion Forecasting in Complex Scenes
  8. 2025Multi-Person Interaction Generation from Two-Person Motion Priors