# Towards AI-Powered Video Assistant Referee System (VARS) for Association Football

arXiv: https://arxiv.org/abs/2407.12483 · HTML: https://mlsanity.com/paper/2407.12483/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 52 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [MViTv2: Improved Multiscale Vision Transformers for Classification and Detection](https://mlsanity.com/paper/2112.01526/) (2021)
- [SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos](https://mlsanity.com/paper/1804.04527/) (2018)
- [SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos](https://mlsanity.com/paper/2011.13367/) (2020)
- [SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos](https://mlsanity.com/paper/2204.06918/) (2022)
- [Jointly Learning to Align and Translate with Transformer Models](https://mlsanity.com/paper/1909.02074/) (2019)
- [Body Part-Based Representation Learning for Occluded Person Re-Identification](https://mlsanity.com/paper/2211.03679/) (2022)
- [SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries](https://mlsanity.com/paper/2304.04565/) (2023)
- [A Context-Aware Loss Function for Action Spotting in Soccer Videos](https://mlsanity.com/paper/1912.01326/) (2019)
- [Spotting Temporally Precise, Fine-Grained Events in Video](https://mlsanity.com/paper/2207.10213/) (2022)
- [VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views](https://mlsanity.com/paper/2304.04617/) (2023)
- [DeepSportradar-v1: Computer Vision Dataset for Sports Understanding with High Quality Annotations](https://mlsanity.com/paper/2208.08190/) (2022)
- [SoccerNet Game State Reconstruction: End-to-End Athlete Tracking and Identification on a Minimap](https://mlsanity.com/paper/2404.11335/) (2024)
- [Temporally-Aware Feature Pooling for Action Spotting in Soccer Broadcasts](https://mlsanity.com/paper/2104.06779/) (2021)
- [Temporally Precise Action Spotting in Soccer Videos Using Dense Detection Anchors](https://mlsanity.com/paper/2205.10450/) (2022)
- [Semi-Supervised Training to Improve Player and Ball Detection in Soccer](https://mlsanity.com/paper/2204.06859/) (2022)
- [X-VARS: Introducing Explainability in Football Refereeing with Multi-Modal Large Language Model](https://mlsanity.com/paper/2404.06332/) (2024)
- [Using Player's Body-Orientation to Model Pass Feasibility in Soccer](https://mlsanity.com/paper/2004.07209/) (2020)
- [Efficient tracking of team sport players with few game-specific annotations](https://mlsanity.com/paper/2204.04049/) (2022)
- [Action Spotting using Dense Detection Anchors Revisited: Submission to the SoccerNet Challenge 2022](https://mlsanity.com/paper/2206.07846/) (2022)
- [Towards Active Learning for Action Spotting in Association Football Videos](https://mlsanity.com/paper/2304.04220/) (2023)
- [A Universal Protocol to Benchmark Camera Calibration for Sports](https://mlsanity.com/paper/2404.09807/) (2024)
- [Multimodal and multiview distillation for real-time player detection on a football field](https://mlsanity.com/paper/2004.07544/) (2020)
- [SoccerNet-Echoes: A Soccer Game Audio Commentary Dataset](https://mlsanity.com/paper/2405.07354/) (2024)
- [MMSys'22 Grand Challenge on AI-based Video Production for Soccer](https://mlsanity.com/paper/2202.01031/) (2022)

## Built on by (most cited first, 8 of 8)

- [SPORTU: A Comprehensive Sports Understanding Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2410.08474/) (2024)
- [SoccerNet 2024 Challenges Results](https://mlsanity.com/paper/2409.10587/) (2024)
- [Deep learning for action spotting in association football videos](https://mlsanity.com/paper/2410.01304/) (2024)
- [SoccerNet 2025 Challenges Results](https://mlsanity.com/paper/2508.19182/) (2025)
- [ProfVLM: A lightweight video-language model for multi-view proficiency estimation](https://mlsanity.com/paper/2509.26278/) (2025)
- [Towards Athlete Fatigue Assessment from Association Football Videos](https://mlsanity.com/paper/2604.05636/) (2026)
- [SoccerNet 2026 Challenges Results](https://mlsanity.com/paper/2607.07320/) (2026)
- [CAS-FD: Contact-Aware Temporal Sampling for Single-View Foul vs Dive Recognition](https://mlsanity.com/paper/2608.17060/) (2026)

## Abstract (arXiv)

Over the past decade, the technology used by referees in football has improved substantially, enhancing the fairness and accuracy of decisions. This progress has culminated in the implementation of the Video Assistant Referee (VAR), an innovation that enables backstage referees to review incidents on the pitch from multiple points of view. However, the VAR is currently limited to professional leagues due to its expensive infrastructure and the lack of referees worldwide. In this paper, we present the semi-automated Video Assistant Referee System (VARS) that leverages the latest findings in multi-view video analysis. VARS sets a new state-of-the-art on the SoccerNet-MVFoul dataset, a multi-view video dataset of football fouls. Our VARS achieves a new state-of-the-art on the SoccerNet-MVFoul dataset by recognizing the type of foul in 50% of instances and the appropriate sanction in 46% of cases. Finally, we conducted a comparative study to investigate human performance in classifying fouls and their corresponding severity and compared these findings to our VARS. The results of our study highlight the potential of our VARS to reach human performance and support football refereeing across all levels of professional and amateur federations.