mlsanityData to 14 Oct 2026

Learning Socio-Temporal Graphs for Multi-Agent Trajectory Prediction

arXiv 2312.14373 ↗First posted 22 Dec 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

In order to predict a pedestrian's trajectory in a crowd accurately, one has to take into account her/his underlying socio-temporal interactions with other pedestrians consistently. Unlike existing work that represents the relevant information separately, partially, or implicitly, we propose a complete representation for it to be fully and explicitly captured and analyzed. In particular, we introduce a Directed Acyclic Graph-based structure, which we term Socio-Temporal Graph (STG), to explicitly capture pair-wise socio-temporal interactions among a group of people across both space and time. Our model is built on a time-varying generative process, whose latent variables determine the structure of the STGs. We design an attention-based model named STGformer that affords an end-to-end pipeline to learn the structure of the STGs for trajectory prediction. Our solution achieves overall state-of-the-art prediction accuracy in two large-scale benchmark datasets. Our analysis shows that a person's past trajectory is critical for predicting another person's future path. Our model learns this relationship with a strong notion of socio-temporal localities. Statistics show that utilizing this information explicitly for prediction yields a noticeable performance gain with respect to the trajectory-only approaches.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 62 references that are in the map. First posted on arXiv 22 Dec 2023.

Builds on

37 of 62 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Graph Attention Networkscited by 8,923
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  4. 2016Variational Graph Auto-Encoderscited by 1,382
  5. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
  6. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 331
  7. 2020It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Predictioncited by 236
  8. 2019Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networkscited by 229
  9. 2020Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Predictioncited by 218
  10. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 212
  11. 2022Stochastic Trajectory Prediction via Motion Indeterminacy Diffusioncited by 185
  12. 2020Transformer Networks for Trajectory Forecastingcited by 181
25 more
  1. 2019Multiple Futures Predictioncited by 178
  2. 2020From Goals, Waypoints & Paths To Long Term Human Trajectory Forecastingcited by 167
  3. 2020PnPNet: End-to-End Perception and Prediction with Tracking in the Loopcited by 144
  4. 2022GroupNet: Multiscale Hypergraph Neural Networks for Trajectory Prediction with Relational Reasoningcited by 128
  5. 2021DiBS: Differentiable Bayesian Structure Learningcited by 106
  6. 2022Remember Intentions: Retrospective-Memory-based Trajectory Predictioncited by 86
  7. 2020Causal Discovery in Physical Systems from Videoscited by 77
  8. 2022Human Trajectory Prediction via Neural Social Physicscited by 66
  9. 2021Stepwise Goal-Driven Networks for Trajectory Predictioncited by 62
  10. 2021Trajectory Prediction with Latent Belief Energy-Based Modelcited by 60
  11. 2022MUSE-VAE: Multi-Scale VAE for Environment-Aware Long Term Trajectory Predictioncited by 59
  12. 2020Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Datacited by 58
  13. 2020Social NCE: Contrastive Learning of Socially-aware Motion Representationscited by 58
  14. 2021Three Steps to Multimodal Trajectory Prediction: Modality Clustering, Classification and Synthesiscited by 46
  15. 2022Non-Probability Sampling Network for Stochastic Human Trajectory Predictioncited by 46
  16. 2021View Vertically: A Hierarchical Network for Trajectory Prediction via Fourier Spectrumscited by 45
  17. 2021Interpretable Social Anchors for Human Trajectory Forecasting in Crowdscited by 44
  18. 2022Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 44
  19. 2022Learning Pedestrian Group Representations for Multi-modal Trajectory Predictioncited by 44
  20. 2021Human Trajectory Prediction via Counterfactual Analysiscited by 42
  21. 2021Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspectivecited by 40
  22. 2021RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecastingcited by 36
  23. 2022Graph-based Spatial Transformer with Memory Replay for Multi-future Pedestrian Trajectory Predictioncited by 36
  24. 2019Generative Hybrid Representations for Activity Forecasting with No-Regret Learningcited by 18
  25. 2019Visual Reaction: Learning to Play Catch with Your Dronecited by 6

Built on by

Not cited yet.