# Foul prediction with estimated poses from soccer broadcast video

arXiv: https://arxiv.org/abs/2402.09650 · HTML: https://mlsanity.com/paper/2402.09650/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 52 references that are in the map.
- First posted on arXiv 15 Feb 2024.

## Builds on (most cited first, 25 of 25)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [MixFormer: End-to-End Tracking with Iterative Mixed Attention](https://mlsanity.com/paper/2203.11082/) (2022)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos](https://mlsanity.com/paper/1804.04527/) (2018)
- [SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos](https://mlsanity.com/paper/2011.13367/) (2020)
- [SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenes](https://mlsanity.com/paper/2304.05170/) (2023)
- [Pose2Seg: Detection Free Human Instance Segmentation](https://mlsanity.com/paper/1803.10683/) (2018)
- [Actions Speak Louder Than Goals: Valuing Player Actions in Soccer](https://mlsanity.com/paper/1802.07127/) (2018)
- [SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos](https://mlsanity.com/paper/2204.06918/) (2022)
- [SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries](https://mlsanity.com/paper/2304.04565/) (2023)
- [A Context-Aware Loss Function for Action Spotting in Soccer Videos](https://mlsanity.com/paper/1912.01326/) (2019)
- [VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views](https://mlsanity.com/paper/2304.04617/) (2023)
- [Temporally-Aware Feature Pooling for Action Spotting in Soccer Broadcasts](https://mlsanity.com/paper/2104.06779/) (2021)
- [Multi-Modal Hybrid Architecture for Pedestrian Action Prediction](https://mlsanity.com/paper/2012.00514/) (2020)
- [FuSSI-Net: Fusion of Spatio-temporal Skeletons for Intention Prediction Network](https://mlsanity.com/paper/2005.07796/) (2020)
- [Evaluation of soccer team defense based on prediction models of ball recovery and being attacked: A pilot study](https://mlsanity.com/paper/2103.09627/) (2021)
- [Towards Active Learning for Action Spotting in Association Football Videos](https://mlsanity.com/paper/2304.04220/) (2023)
- [Transformer-Based Neural Marked Spatio Temporal Point Process Model for Football Match Events Analysis](https://mlsanity.com/paper/2302.09276/) (2023)
- [Location analysis of players in UEFA EURO 2020 and 2022 using generalized valuation of defense by estimating probabilities](https://mlsanity.com/paper/2212.00021/) (2022)
- [Soccer Event Detection Using Deep Learning](https://mlsanity.com/paper/2102.04331/) (2021)

## Built on by (most cited first, 7 of 7)

- [X-VARS: Introducing Explainability in Football Refereeing with Multi-Modal Large Language Model](https://mlsanity.com/paper/2404.06332/) (2024)
- [AutoSoccerPose: Automated 3D posture Analysis of Soccer Shot Movements](https://mlsanity.com/paper/2405.12070/) (2024)
- [AthletePose3D: A Benchmark Dataset for 3D Human Pose Estimation and Kinematic Validation in Athletic Movements](https://mlsanity.com/paper/2503.07499/) (2025)
- [SoccerNet 2025 Challenges Results](https://mlsanity.com/paper/2508.19182/) (2025)
- [SoccerSynth-Detection: A Synthetic Dataset for Soccer Player Detection](https://mlsanity.com/paper/2501.09281/) (2025)
- [Towards Athlete Fatigue Assessment from Association Football Videos](https://mlsanity.com/paper/2604.05636/) (2026)
- [SoccerNet 2026 Challenges Results](https://mlsanity.com/paper/2607.07320/) (2026)

## Abstract (arXiv)

Recent advances in computer vision have made significant progress in tracking and pose estimation of sports players. However, there have been fewer studies on behavior prediction with pose estimation in sports, in particular, the prediction of soccer fouls is challenging because of the smaller image size of each player and of difficulty in the usage of e.g., the ball and pose information. In our research, we introduce an innovative deep learning approach for anticipating soccer fouls. This method integrates video data, bounding box positions, image details, and pose information by curating a novel soccer foul dataset. Our model utilizes a combination of convolutional and recurrent neural networks (CNNs and RNNs) to effectively merge information from these four modalities. The experimental results show that our full model outperformed the ablated models, and all of the RNN modules, bounding box position and image, and estimated pose were useful for the foul prediction. Our findings have important implications for a deeper understanding of foul play in soccer and provide a valuable reference for future research and practice in this area.