# Ball Trajectory Inference from Multi-Agent Sports Contexts Using Set Transformer and Hierarchical Bi-LSTM

arXiv: https://arxiv.org/abs/2306.08206 · HTML: https://mlsanity.com/paper/2306.08206/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 49 references that are in the map.
- First posted on arXiv 14 Jun 2023.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [InceptionTime: Finding AlexNet for Time Series Classification](https://mlsanity.com/paper/1909.04939/) (2019)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [BRITS: Bidirectional Recurrent Imputation for Time Series](https://mlsanity.com/paper/1805.10572/) (2018)
- [Actions Speak Louder Than Goals: Valuing Player Actions in Soccer](https://mlsanity.com/paper/1802.07127/) (2018)
- [NAOMI: Non-Autoregressive Multiresolution Sequence Imputation](https://mlsanity.com/paper/1901.10946/) (2019)
- [Generating Multi-Agent Trajectories using Programmatic Weak Supervision](https://mlsanity.com/paper/1803.07612/) (2018)
- [STINet: Spatio-Temporal-Interactive Network for Pedestrian Detection and Trajectory Prediction](https://mlsanity.com/paper/2005.04255/) (2020)
- [What Players do with the Ball: A Physically Constrained Interaction Modeling](https://mlsanity.com/paper/1511.06181/) (2015)
- [PlayeRank: data-driven performance evaluation and player ranking in soccer via a machine learning approach](https://mlsanity.com/paper/1802.04987/) (2018)
- [Automatic event detection in football using tracking data](https://mlsanity.com/paper/2202.00804/) (2022)
- [Player Chemistry: Striving for a Perfectly Balanced Soccer Team](https://mlsanity.com/paper/2003.01712/) (2020)
- [Smart Director: An Event-Driven Directing System for Live Broadcasting](https://mlsanity.com/paper/2201.04024/) (2022)
- [Automatic Pass Annotation from Soccer VideoStreams Based on Object Detection and LSTM](https://mlsanity.com/paper/2007.06475/) (2020)

## Built on by (most cited first, 12 of 12)

- [Unified Uncertainty-Aware Diffusion for Multi-Agent Trajectory Modeling](https://mlsanity.com/paper/2503.18589/) (2025)
- [TranSPORTmer: A Holistic Approach to Trajectory Understanding in Multi-Agent Sports](https://mlsanity.com/paper/2410.17785/) (2024)
- [Improving Object Detection Quality in Football Through Super-Resolution Techniques](https://mlsanity.com/paper/2402.00163/) (2024)
- [JointDiff: Bridging Continuous and Discrete in Multi-Agent Trajectory Generation](https://mlsanity.com/paper/2509.22522/) (2025)
- [Trajectory Imputation in Multi-Agent Sports with Derivative-Accumulating Self-Ensemble](https://mlsanity.com/paper/2408.10878/) (2024)
- [Heteroscedastic Diffusion for Multi-Agent Trajectory Modeling](https://mlsanity.com/paper/2605.10717/) (2026)
- [Contextual Sprint Classification in Soccer Based on Deep Learning](https://mlsanity.com/paper/2406.15659/) (2024)
- [Multi-Modal Soccer Scene Analysis with Masked Pre-Training](https://mlsanity.com/paper/2512.19528/) (2025)
- [Modeling Matches as Language: A Generative Transformer Approach for Counterfactual Player Valuation in Football](https://mlsanity.com/paper/2603.15212/) (2026)
- [SoccerTrack v2: A Full-Pitch Panoramic Video Dataset for Game State Reconstruction and Ball Action Spotting](https://mlsanity.com/paper/2508.01802/) (2025)
- [PathCRF: Ball-Free Soccer Event Detection via Possession Path Inference from Player Trajectories](https://mlsanity.com/paper/2602.12080/) (2026)
- [TacSIm: A Dataset and Benchmark for Football Tactical Style Imitation](https://mlsanity.com/paper/2603.25199/) (2026)

## Abstract (arXiv)

As artificial intelligence spreads out to numerous fields, the application of AI to sports analytics is also in the spotlight. However, one of the major challenges is the difficulty of automated acquisition of continuous movement data during sports matches. In particular, it is a conundrum to reliably track a tiny ball on a wide soccer pitch with obstacles such as occlusion and imitations. Tackling the problem, this paper proposes an inference framework of ball trajectory from player trajectories as a cost-efficient alternative to ball tracking. We combine Set Transformers to get permutation-invariant and equivariant representations of the multi-agent contexts with a hierarchical architecture that intermediately predicts the player ball possession to support the final trajectory inference. Also, we introduce the reality loss term and postprocessing to secure the estimated trajectories to be physically realistic. The experimental results show that our model provides natural and accurate trajectories as well as admissible player ball possession at the same time. Lastly, we suggest several practical applications of our framework including missing trajectory imputation, semi-automated pass annotation, automated zoom-in for match broadcasting, and calculating possession-wise running performance metrics.