# VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views

arXiv: https://arxiv.org/abs/2304.04617 · HTML: https://mlsanity.com/paper/2304.04617/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 83 references that are in the map.
- First posted on arXiv 10 Apr 2023.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [MViTv2: Improved Multiscale Vision Transformers for Classification and Detection](https://mlsanity.com/paper/2112.01526/) (2021)
- [YouTube-8M: A Large-Scale Video Classification Benchmark](https://mlsanity.com/paper/1609.08675/) (2016)
- [Temporal Segment Networks for Action Recognition in Videos](https://mlsanity.com/paper/1705.02953/) (2017)
- [Beyond Short Snippets: Deep Networks for Video Classification](https://mlsanity.com/paper/1503.08909/) (2015)
- [Video Generation From Text](https://mlsanity.com/paper/1710.00421/) (2017)
- [SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos](https://mlsanity.com/paper/1804.04527/) (2018)
- [MVTN: Multi-View Transformation Network for 3D Shape Recognition](https://mlsanity.com/paper/2011.13244/) (2020)
- [Cross-view Action Modeling, Learning and Recognition](https://mlsanity.com/paper/1405.2941/) (2014)
- [SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos](https://mlsanity.com/paper/2011.13367/) (2020)
- [Actions Speak Louder Than Goals: Valuing Player Actions in Soccer](https://mlsanity.com/paper/1802.07127/) (2018)
- [SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos](https://mlsanity.com/paper/2204.06918/) (2022)
- [Body Part-Based Representation Learning for Occluded Person Re-Identification](https://mlsanity.com/paper/2211.03679/) (2022)
- [A Context-Aware Loss Function for Action Spotting in Soccer Videos](https://mlsanity.com/paper/1912.01326/) (2019)
- [Video Captioning via Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1711.11135/) (2017)
- [Equivariant Multi-View Networks](https://mlsanity.com/paper/1904.00993/) (2019)
- [Spotting Temporally Precise, Fine-Grained Events in Video](https://mlsanity.com/paper/2207.10213/) (2022)
- [DeepSportradar-v1: Computer Vision Dataset for Sports Understanding with High Quality Annotations](https://mlsanity.com/paper/2208.08190/) (2022)
- [Unsupervised Learning of View-invariant Action Representations](https://mlsanity.com/paper/1809.01844/) (2018)
- [A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications](https://mlsanity.com/paper/2206.01038/) (2022)
- [SoccerNet 2022 Challenges Results](https://mlsanity.com/paper/2210.02365/) (2022)

## Built on by (most cited first, 30 of 38)

- [MatchTime: Towards Automatic Soccer Game Commentary Generation](https://mlsanity.com/paper/2406.18530/) (2024)
- [SoccerNet Game State Reconstruction: End-to-End Athlete Tracking and Identification on a Minimap](https://mlsanity.com/paper/2404.11335/) (2024)
- [SPORTU: A Comprehensive Sports Understanding Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2410.08474/) (2024)
- [X-VARS: Introducing Explainability in Football Refereeing with Multi-Modal Large Language Model](https://mlsanity.com/paper/2404.06332/) (2024)
- [Towards Universal Soccer Video Understanding](https://mlsanity.com/paper/2412.01820/) (2024)
- [SoccerNet 2023 Challenges Results](https://mlsanity.com/paper/2309.06006/) (2023)
- [AutoSoccerPose: Automated 3D posture Analysis of Soccer Shot Movements](https://mlsanity.com/paper/2405.12070/) (2024)
- [A Universal Protocol to Benchmark Camera Calibration for Sports](https://mlsanity.com/paper/2404.09807/) (2024)
- [Multi-task Learning for Joint Re-identification, Team Affiliation, and Role Classification for Sports Visual Tracking](https://mlsanity.com/paper/2401.09942/) (2024)
- [SoccerNet-Echoes: A Soccer Game Audio Commentary Dataset](https://mlsanity.com/paper/2405.07354/) (2024)
- [SoccerMaster: A Vision Foundation Model for Soccer Understanding](https://mlsanity.com/paper/2512.11016/) (2025)
- [Action Anticipation from SoccerNet Football Video Broadcasts](https://mlsanity.com/paper/2504.12021/) (2025)
- [Towards AI-Powered Video Assistant Referee System (VARS) for Association Football](https://mlsanity.com/paper/2407.12483/) (2024)
- [SoccerNet 2024 Challenges Results](https://mlsanity.com/paper/2409.10587/) (2024)
- [Foul prediction with estimated poses from soccer broadcast video](https://mlsanity.com/paper/2402.09650/) (2024)
- [3D Pose-Based Temporal Action Segmentation for Figure Skating: A Fine-Grained and Jump Procedure-Aware Annotation Approach](https://mlsanity.com/paper/2408.16638/) (2024)
- [Deep learning for action spotting in association football videos](https://mlsanity.com/paper/2410.01304/) (2024)
- [Domain Adaptation of VLM for Soccer Video Understanding](https://mlsanity.com/paper/2505.13860/) (2025)
- [UKBOB: One Billion MRI Labeled Masks for Generalizable 3D Medical Image Segmentation](https://mlsanity.com/paper/2504.06908/) (2025)
- [SoccerNet-v3D: Leveraging Sports Broadcast Replays for 3D Scene Understanding](https://mlsanity.com/paper/2504.10106/) (2025)
- [SoccerNet 2025 Challenges Results](https://mlsanity.com/paper/2508.19182/) (2025)
- [SV3.3B: A Sports Video Understanding Model for Action Recognition](https://mlsanity.com/paper/2507.17844/) (2025)
- [OSL-ActionSpotting: A Unified Library for Action Spotting in Sports Videos](https://mlsanity.com/paper/2407.01265/) (2024)
- [Action Valuation in Sports: A Survey](https://mlsanity.com/paper/2504.06163/) (2025)
- [Can Geometry Save Central Views for Sports Field Registration?](https://mlsanity.com/paper/2504.20052/) (2025)
- [Investigating Event-Based Cameras for Video Frame Interpolation in Sports](https://mlsanity.com/paper/2407.02370/) (2024)
- [Multi Camera Connected Vision System with Multi View Analytics: A Comprehensive Survey](https://mlsanity.com/paper/2510.09731/) (2025)
- [Towards Athlete Fatigue Assessment from Association Football Videos](https://mlsanity.com/paper/2604.05636/) (2026)
- [RefereeBench: Are Video MLLMs Ready to be Multi-Sport Referees](https://mlsanity.com/paper/2604.15736/) (2026)
- [SoccerLens: Grounded Soccer Video Understanding Beyond Accuracy](https://mlsanity.com/paper/2605.09598/) (2026)

## Abstract (arXiv)

The Video Assistant Referee (VAR) has revolutionized association football, enabling referees to review incidents on the pitch, make informed decisions, and ensure fairness. However, due to the lack of referees in many countries and the high cost of the VAR infrastructure, only professional leagues can benefit from it. In this paper, we propose a Video Assistant Referee System (VARS) that can automate soccer decision-making. VARS leverages the latest findings in multi-view video analysis, to provide real-time feedback to the referee, and help them make informed decisions that can impact the outcome of a game. To validate VARS, we introduce SoccerNet-MVFoul, a novel video dataset of soccer fouls from multiple camera views, annotated with extensive foul descriptions by a professional soccer referee, and we benchmark our VARS to automatically recognize the characteristics of these fouls. We believe that VARS has the potential to revolutionize soccer refereeing and take the game to new heights of fairness and accuracy across all levels of professional and amateur federations.