# A Foundation Model for Soccer

arXiv: https://arxiv.org/abs/2407.14558 · HTML: https://mlsanity.com/paper/2407.14558/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 12 references that are in the map.
- First posted on arXiv 18 Jul 2024.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Actions Speak Louder Than Goals: Valuing Player Actions in Soccer](https://mlsanity.com/paper/1802.07127/) (2018)
- [A framework for the fine-grained evaluation of the instantaneous expected value of soccer possessions](https://mlsanity.com/paper/2011.09426/) (2020)
- [Forecasting Events in Soccer Matches Through Language](https://mlsanity.com/paper/2402.06820/) (2024)

## Built on by (most cited first, 3 of 3)

- [OpenSTARLab: Open Approach for Spatio-Temporal Agent Data Analysis in Soccer](https://mlsanity.com/paper/2502.02785/) (2025)
- [Beyond Pixels: Leveraging the Language of Soccer to Improve Spatio-Temporal Action Detection in Broadcast Videos](https://mlsanity.com/paper/2505.09455/) (2025)
- [Modeling Matches as Language: A Generative Transformer Approach for Counterfactual Player Valuation in Football](https://mlsanity.com/paper/2603.15212/) (2026)

## Abstract (arXiv)

We propose a foundation model for soccer, which is able to predict subsequent actions in a soccer match from a given input sequence of actions. As a proof of concept, we train a transformer architecture on three seasons of data from a professional soccer league. We quantitatively and qualitatively compare the performance of this transformer architecture to two baseline models: a Markov model and a multi-layer perceptron. Additionally, we discuss potential applications of our model. We provide an open-source implementation of our methods at https://github.com/danielhocevar/Foundation-Model-for-Soccer.