Stochastic Prediction of Multi-Agent Interactions from Partial Observations
Not on the map. Why?
Abstract (arXiv)
We present a method that learns to integrate temporal information, from a learned dynamics model, with ambiguous visual information, from a learned vision model, in the context of interacting agents. Our method is based on a graph-structured variational recurrent neural network (Graph-VRNN), which is trained end-to-end to infer the current state of the (partially observed) world, as well as to forecast future states. We show that our method outperforms various baselines on two sports datasets, one based on real basketball trajectories, and one generated by a soccer game engine.
Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Feb 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 7; 2022: 9; 2023: 5; 2024: 3; 2025: 1; 2026: 1.
26 papers.
Sort:
By area:
- 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
- 2022GraphCast: Learning skillful medium-range global weather forecastingcited by 261
- 2021Towards Causal Representation Learningcited by 207
- 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
- 2021ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representationscited by 67
- 2021Spatio-Temporal Dynamic Inference Network for Group Activity Recognitioncited by 42
- 2022How many Observations are Enough? Knowledge Distillation for Trajectory Forecastingcited by 35
- 2021CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoningcited by 23
- 2023Neural Relational Inference with Fast Modular Meta-learningcited by 16
- 2025Unified Uncertainty-Aware Diffusion for Multi-Agent Trajectory Modelingcited by 15
- 2021Learning Temporal Dynamics from Cycles in Narrated Videocited by 13
- 2023Object-Centric Learning with Slot Mixture Modulecited by 11
14 more
- 2024TranSPORTmer: A Holistic Approach to Trajectory Understanding in Multi-Agent Sportscited by 11
- 2023Inferring Player Location in Sports Matches: Multi-Agent Spatial Imputation from Limited Observationscited by 9
- 2022Aware of the History: Trajectory Forecasting with the Local Behavior Datacited by 8
- 2024FootBots: A Transformer-based Architecture for Motion Prediction in Soccercited by 8
- 2021Agent-Centric Representations for Multi-Agent Reinforcement Learningcited by 6
- 2022Group Activity Recognition in Basketball Tracking Data -- Neural Embeddings in Team Sports (NETS)cited by 5
- 2022Representing Spatial Trajectories as Distributionscited by 5
- 2023Co-Learning Empirical Games and World Modelscited by 4
- 2022Dance of SNN and ANN: Solving binding problem by combining spike timing and reconstructive attentioncited by 3
- 2026Heteroscedastic Diffusion for Multi-Agent Trajectory Modelingcited by 3
- 2023About latent roles in forecasting players in team sportscited by 2
- 2024Trajectory Forecasting through Low-Rank Adaptation of Discrete Latent Codescited by 2
- 2022Modeling Human Behavior Part I -- Learning and Belief Approachescited by 1
- 2022Rethinking Trajectory Prediction via "Team Game"cited by 1
- 2026Heteroscedastic Diffusion for Multi-Agent Trajectory Modeling11 May 2026
- 2025Unified Uncertainty-Aware Diffusion for Multi-Agent Trajectory Modeling24 Mar 2025
- 2024TranSPORTmer: A Holistic Approach to Trajectory Understanding in Multi-Agent Sports23 Oct 2024
- 2024FootBots: A Transformer-based Architecture for Motion Prediction in Soccer28 Jun 2024
- 2024Trajectory Forecasting through Low-Rank Adaptation of Discrete Latent Codes31 May 2024
- 2023Object-Centric Learning with Slot Mixture Module8 Nov 2023
- 2023Neural Relational Inference with Fast Modular Meta-learning10 Oct 2023
- 2023Co-Learning Empirical Games and World Models23 May 2023
- 2023About latent roles in forecasting players in team sports17 Apr 2023
- 2023Inferring Player Location in Sports Matches: Multi-Agent Spatial Imputation from Limited Observations13 Feb 2023
- 2022GraphCast: Learning skillful medium-range global weather forecasting24 Dec 2022
- 2022Dance of SNN and ANN: Solving binding problem by combining spike timing and reconstructive attention11 Nov 2022
14 more
- 2022Rethinking Trajectory Prediction via "Team Game"17 Oct 2022
- 2022Representing Spatial Trajectories as Distributions4 Oct 2022
- 2022Group Activity Recognition in Basketball Tracking Data -- Neural Embeddings in Team Sports (NETS)31 Aug 2022
- 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queries2 Aug 2022
- 2022Aware of the History: Trajectory Forecasting with the Local Behavior Data20 Jul 2022
- 2022Modeling Human Behavior Part I -- Learning and Belief Approaches13 May 2022
- 2022How many Observations are Enough? Knowledge Distillation for Trajectory Forecasting9 Mar 2022
- 2021ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations16 Sep 2021
- 2021Spatio-Temporal Dynamic Inference Network for Group Activity Recognition26 Aug 2021
- 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets22 Aug 2021
- 2021CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning25 Jun 2021
- 2021Agent-Centric Representations for Multi-Agent Reinforcement Learning19 Apr 2021
- 2021Towards Causal Representation Learning22 Feb 2021
- 2021Learning Temporal Dynamics from Cycles in Narrated Video7 Jan 2021
- 2023Co-Learning Empirical Games and World Modelscited by 4
- 2023Co-Learning Empirical Games and World Models23 May 2023
- 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
- 2022GraphCast: Learning skillful medium-range global weather forecastingcited by 261
- 2021Towards Causal Representation Learningcited by 207
- 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
- 2021ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representationscited by 67
- 2021Spatio-Temporal Dynamic Inference Network for Group Activity Recognitioncited by 42
- 2022How many Observations are Enough? Knowledge Distillation for Trajectory Forecastingcited by 35
- 2021CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoningcited by 23
- 2023Neural Relational Inference with Fast Modular Meta-learningcited by 16
- 2025Unified Uncertainty-Aware Diffusion for Multi-Agent Trajectory Modelingcited by 15
- 2021Learning Temporal Dynamics from Cycles in Narrated Videocited by 13
- 2023Object-Centric Learning with Slot Mixture Modulecited by 11
12 most cited of 24 All 24 (JSON)
- 2025Unified Uncertainty-Aware Diffusion for Multi-Agent Trajectory Modeling24 Mar 2025
- 2024TranSPORTmer: A Holistic Approach to Trajectory Understanding in Multi-Agent Sports23 Oct 2024
- 2024FootBots: A Transformer-based Architecture for Motion Prediction in Soccer28 Jun 2024
- 2024Trajectory Forecasting through Low-Rank Adaptation of Discrete Latent Codes31 May 2024
- 2023Object-Centric Learning with Slot Mixture Module8 Nov 2023
- 2023Neural Relational Inference with Fast Modular Meta-learning10 Oct 2023
- 2023About latent roles in forecasting players in team sports17 Apr 2023
- 2023Inferring Player Location in Sports Matches: Multi-Agent Spatial Imputation from Limited Observations13 Feb 2023
- 2022GraphCast: Learning skillful medium-range global weather forecasting24 Dec 2022
- 2022Dance of SNN and ANN: Solving binding problem by combining spike timing and reconstructive attention11 Nov 2022
- 2022Rethinking Trajectory Prediction via "Team Game"17 Oct 2022
- 2022Representing Spatial Trajectories as Distributions4 Oct 2022
12 newest of 24 All 24 (JSON)