# Time-series Imputation of Temporally-occluded Multiagent Trajectories

arXiv: https://arxiv.org/abs/2106.04219 · HTML: https://mlsanity.com/paper/2106.04219/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 42 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [BRITS: Bidirectional Recurrent Imputation for Time Series](https://mlsanity.com/paper/1805.10572/) (2018)
- [Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019](https://mlsanity.com/paper/1911.13288/) (2019)
- [Human Motion Trajectory Prediction: A Survey](https://mlsanity.com/paper/1905.06113/) (2019)
- [Deep Counterfactual Regret Minimization](https://mlsanity.com/paper/1811.00164/) (2018)
- [MaskGAN: Better Text Generation via Filling in the______](https://mlsanity.com/paper/1801.07736/) (2018)
- [Estimating Missing Data in Temporal Data Streams Using Multi-directional Recurrent Neural Networks](https://mlsanity.com/paper/1711.08742/) (2017)
- [Coordinated Multi-Agent Imitation Learning](https://mlsanity.com/paper/1703.03121/) (2017)
- [NAOMI: Non-Autoregressive Multiresolution Sequence Imputation](https://mlsanity.com/paper/1901.10946/) (2019)
- [VAIN: Attentional Multi-agent Predictive Modeling](https://mlsanity.com/paper/1706.06122/) (2017)
- [Reciprocal Learning Networks for Human Trajectory Prediction](https://mlsanity.com/paper/2004.04340/) (2020)
- [Multi-Modal Trajectory Prediction of NBA Players](https://mlsanity.com/paper/2008.07870/) (2020)
- [baller2vec: A Multi-Entity Transformer For Multi-Agent Spatiotemporal Modeling](https://mlsanity.com/paper/2102.03291/) (2021)
- [Learning Interaction-Aware Trajectory Predictions for Decentralized Multi-Robot Motion Planning in Dynamic Environments](https://mlsanity.com/paper/2102.05382/) (2021)
- [baller2vec++: A Look-Ahead Multi-Entity Transformer For Modeling Coordinated Agents](https://mlsanity.com/paper/2104.11980/) (2021)

## Built on by (most cited first, 7 of 7)

- [Sports-Traj: A Unified Trajectory Generation Model for Multi-Agent Movement in Sports](https://mlsanity.com/paper/2405.17680/) (2024)
- [Centralized Training with Hybrid Execution in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2210.06274/) (2022)
- [Layout Sequence Prediction From Noisy Mobile Modality](https://mlsanity.com/paper/2310.06138/) (2023)
- [Multi-Modal Soccer Scene Analysis with Masked Pre-Training](https://mlsanity.com/paper/2512.19528/) (2025)
- [AdaSports-Traj: Role- and Domain-Aware Adaptation for Multi-Agent Trajectory Modeling in Sports](https://mlsanity.com/paper/2509.16095/) (2025)
- [TacSIm: A Dataset and Benchmark for Football Tactical Style Imitation](https://mlsanity.com/paper/2603.25199/) (2026)
- [Monte Carlo Pass Search: Using Trajectory Generation for 3D Counterfactual Pass Evaluation in Football](https://mlsanity.com/paper/2606.11120/) (2026)

## Abstract (arXiv)

In multiagent environments, several decision-making individuals interact while adhering to the dynamics constraints imposed by the environment. These interactions, combined with the potential stochasticity of the agents' decision-making processes, make such systems complex and interesting to study from a dynamical perspective. Significant research has been conducted on learning models for forward-direction estimation of agent behaviors, for example, pedestrian predictions used for collision-avoidance in self-driving cars. However, in many settings, only sporadic observations of agents may be available in a given trajectory sequence. For instance, in football, subsets of players may come in and out of view of broadcast video footage, while unobserved players continue to interact off-screen. In this paper, we study the problem of multiagent time-series imputation, where available past and future observations of subsets of agents are used to estimate missing observations for other agents. Our approach, called the Graph Imputer, uses forward- and backward-information in combination with graph networks and variational autoencoders to enable learning of a distribution of imputed trajectories. We evaluate our approach on a dataset of football matches, using a projective camera module to train and evaluate our model for the off-screen player state estimation setting. We illustrate that our method outperforms several state-of-the-art approaches, including those hand-crafted for football.