mlsanityData to 7 Oct 2026

MATRIX: Multi-Agent Trajectory Generation with Diverse Contexts

arXiv 2403.06041 ↗First posted 9 Mar 2024cs.RO cs.AI cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Data-driven methods have great advantages in modeling complicated human behavioral dynamics and dealing with many human-robot interaction applications. However, collecting massive and annotated real-world human datasets has been a laborious task, especially for highly interactive scenarios. On the other hand, algorithmic data generation methods are usually limited by their model capacities, making them unable to offer realistic and diverse data needed by various application users. In this work, we study trajectory-level data generation for multi-human or human-robot interaction scenarios and propose a learning-based automatic trajectory generation model, which we call Multi-Agent TRajectory generation with dIverse conteXts (MATRIX). MATRIX is capable of generating interactive human behaviors in realistic diverse contexts. We achieve this goal by modeling the explicit and interpretable objectives so that MATRIX can generate human motions based on diverse destinations and heterogeneous behaviors. We carried out extensive comparison and ablation studies to illustrate the effectiveness of our approach across various metrics. We also presented experiments that demonstrate the capability of MATRIX to serve as data augmentation for imitation-based motion planning.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 44 of its 60 references that are in the map. First posted on arXiv 9 Mar 2024.

Builds on

44 of 60 references.

References · 44

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
  3. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
  4. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  5. 2020TNT: Target-driveN Trajectory Predictioncited by 345
  6. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 327
  7. 2020Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Predictioncited by 324
  8. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  9. 2018SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraintscited by 296
  10. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
  11. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  12. 2020It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Predictioncited by 235
32 more
  1. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
  2. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 187
  3. 2017Social Attention: Modeling Attention in Human Crowdscited by 185
  4. 2022Stochastic Trajectory Prediction via Motion Indeterminacy Diffusioncited by 184
  5. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
  6. 2020From Goals, Waypoints & Paths To Long Term Human Trajectory Forecastingcited by 166
  7. 2018Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learningcited by 162
  8. 2023Leapfrog Diffusion Model for Stochastic Trajectory Predictioncited by 129
  9. 2022BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Predictioncited by 82
  10. 2019Diverse Trajectory Forecasting with Determinantal Point Processescited by 80
  11. 2018Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMscited by 77
  12. 2019Conditional Generative Neural System for Probabilistic Trajectory Predictioncited by 76
  13. 2022ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planningcited by 63
  14. 2021LOKI: Long Term and Key Intentions for Trajectory Predictioncited by 57
  15. 2022Non-Probability Sampling Network for Stochastic Human Trajectory Predictioncited by 46
  16. 2019Relational Graph Learning for Crowd Navigationcited by 42
  17. 2021Human Trajectory Prediction via Counterfactual Analysiscited by 42
  18. 2021RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecastingcited by 36
  19. 2021Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Trackingcited by 35
  20. 2020Shared Cross-Modal Trajectory Prediction for Autonomous Drivingcited by 31
  21. 2024Generative Expressive Robot Behaviors using Large Language Modelscited by 22
  22. 2024Multi-Agent Dynamic Relational Reasoning for Social Robot Navigationcited by 20
  23. 2021Grouptron: Dynamic Multi-Scale Graph Convolutional Networks for Group-Aware Dense Crowd Trajectory Forecastingcited by 19
  24. 2019Interaction-aware Multi-agent Tracking and Probabilistic Behavior Prediction via Adversarial Learningcited by 15
  25. 2022Interaction Modeling with Multiplex Attentioncited by 14
  26. 2020End-to-end Autonomous Driving Perception with Sequential Latent Representation Learningcited by 11
  27. 2022Holistic Transformer: A Joint Neural Network for Trajectory Prediction and Decision-Making of Autonomous Vehiclescited by 8
  28. 2024Disentangled Neural Relational Inference for Interpretable Motion Predictioncited by 7
  29. 2020Guided Policy Search Model-based Reinforcement Learning for Urban Autonomous Drivingcited by 5
  30. 2020Cascade Attribute Network: Decomposing Reinforcement Learning Control Policies using Hierarchical Neural Networkscited by 4
  31. 2023Distributed Multi-agent Interaction Generation with Imagined Potential Gamescited by 2
  32. 2021History Encoding Representation Design for Human Intention Inferencecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 2.

9 papers.

  1. 2024Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphscited by 72
  2. 2025UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Drivingcited by 27
  3. 2026Graph-based Agent Memory: Taxonomy, Techniques, and Applicationscited by 22
  4. 2025Physical Plausibility-aware Trajectory Prediction via Locomotion Embodimentcited by 10
  5. 2025TrajEvo: Trajectory Prediction Heuristics Design via LLM-driven Evolutioncited by 5
  6. 2025Towards Generalizable Safety in Crowd Navigation via Conformal Uncertainty Handlingcited by 5
  7. 2024Annealed Winner-Takes-All for Motion Forecastingcited by 4
  8. 2025Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Reviewcited by 2
  9. 2026Learning-Accelerated Optimization-based Trajectory Planning for Cooperative Aerial-Ground Handover Missionscited by 0