mlsanityData to 14 Oct 2026

Interaction Modeling with Multiplex Attention

arXiv 2208.10660 ↗First posted 23 Aug 2022cs.LG cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Modeling multi-agent systems requires understanding how agents interact. Such systems are often difficult to model because they can involve a variety of types of interactions that layer together to drive rich social behavioral dynamics. Here we introduce a method for accurately modeling multi-agent systems. We present Interaction Modeling with Multiplex Attention (IMMA), a forward prediction model that uses a multiplex latent graph to represent multiple independent types of interactions and attention to account for relations of different strengths. We also introduce Progressive Layer Training, a training strategy for this architecture. We show that our approach outperforms state-of-the-art models in trajectory forecasting and relation inference, spanning three multi-agent scenarios: social navigation, cooperative task achievement, and team sports. We further demonstrate that our approach can improve zero-shot generalization and allows us to probe how different interactions impact agent behavior.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 29 of its 51 references that are in the map. First posted on arXiv 23 Aug 2022.

Builds on

29 of 51 references.

References · 29

  1. 2017Graph Attention Networkscited by 8,923
  2. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  3. 2018Relational inductive biases, deep learning, and graph networkscited by 1,360
  4. 2020Learning to Simulate Complex Physics with Graph Networkscited by 880
  5. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
  6. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 509
  7. 2018Neural Relational Inference for Interacting Systemscited by 339
  8. 2020Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Predictioncited by 325
  9. 2021Spectral Temporal Graph Neural Network for Multivariate Time-series Forecastingcited by 299
  10. 2017Stochastic Variational Video Predictioncited by 234
  11. 2019Human Motion Trajectory Prediction: A Surveycited by 232
  12. 2018Machine Theory of Mindcited by 226
17 more
  1. 2018Convolutional Social Pooling for Vehicle Trajectory Predictioncited by 221
  2. 2017Social Attention: Modeling Attention in Human Crowdscited by 186
  3. 2018Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learningcited by 175
  4. 2019Conditional Generative Neural System for Probabilistic Trajectory Predictioncited by 76
  5. 2017Coordinated Multi-Agent Imitation Learningcited by 72
  6. 2020World Model as a Graph: Learning Latent Landmarks for Planningcited by 69
  7. 2019Relational Graph Learning for Crowd Navigationcited by 48
  8. 2018Generating Multi-Agent Trajectories using Programmatic Weak Supervisioncited by 46
  9. 2021Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Trackingcited by 35
  10. 2020Progressive Learning and Disentanglement of Hierarchical Representationscited by 30
  11. 2017Generating Long-term Trajectories Using Deep Hierarchical Networkscited by 23
  12. 2021PHASE: PHysically-grounded Abstract Social Events for Machine Social Perceptioncited by 22
  13. 2018Relational Forward Models for Multi-Agent Learningcited by 21
  14. 2020Multi-Modal Trajectory Prediction of NBA Playerscited by 19
  15. 2019Factorised Neural Relational Inference for Multi-Interaction Systemscited by 15
  16. 2019Learning World Graphs to Accelerate Hierarchical Reinforcement Learningcited by 12
  17. 2016Disentangled Representations in Neural Modelscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 6; 2025: 3; 2026: 1.

14 papers.

Sort:

By area:

  1. 2025UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Drivingcited by 28
  2. 2025Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directionscited by 14
  3. 2023Pedestrian Crossing Action Recognition and Trajectory Prediction with 3D Human Keypointscited by 12
  4. 2024MATRIX: Multi-Agent Trajectory Generation with Diverse Contextscited by 9
  5. 2023EqDrive: Efficient Equivariant Motion Forecasting with Multi-Modality for Autonomous Drivingcited by 7
  6. 2024Disentangled Neural Relational Inference for Interpretable Motion Predictioncited by 7
  7. 2023Equivariant Map and Agent Geometry for Autonomous Driving Motion Predictioncited by 6
  8. 2025Towards Generalizable Safety in Crowd Navigation via Conformal Uncertainty Handlingcited by 6
  9. 2022Learning Heterogeneous Interaction Strengths by Trajectory Prediction with Graph Neural Networkcited by 4
  10. 2024Trajectory Imputation in Multi-Agent Sports with Derivative-Accumulating Self-Ensemblecited by 3
  11. 2024CHASE: Learning Convex Hull Adaptive Shift for Skeleton-based Multi-Entity Action Recognitioncited by 3
  12. 2024CCF: Cross Correcting Framework for Pedestrian Trajectory Predictioncited by 2
2 more
  1. 2024SSL-Interactions: Pretext Tasks for Interactive Trajectory Predictioncited by 1
  2. 2026PathCRF: Ball-Free Soccer Event Detection via Possession Path Inference from Player Trajectoriescited by 0