# GOAL: A Challenging Knowledge-grounded Video Captioning Benchmark for Real-time Soccer Commentary Generation

arXiv: https://arxiv.org/abs/2303.14655 · HTML: https://mlsanity.com/paper/2303.14655/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 46 references that are in the map.
- First posted on arXiv 26 Mar 2023.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Towards Automatic Learning of Procedures from Web Instructional Videos](https://mlsanity.com/paper/1703.09788/) (2017)
- [VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding](https://mlsanity.com/paper/2109.14084/) (2021)
- [Revisiting the "Video" in Video-Language Understanding](https://mlsanity.com/paper/2206.01720/) (2022)
- [SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning](https://mlsanity.com/paper/2111.13196/) (2021)
- [Sequence to Sequence -- Video to Text](https://mlsanity.com/paper/1505.00487/) (2015)
- [Align and Prompt: Video-and-Language Pre-training with Entity Prompts](https://mlsanity.com/paper/2112.09583/) (2021)
- [SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos](https://mlsanity.com/paper/2011.13367/) (2020)
- [End-to-end Generative Pretraining for Multimodal Video Captioning](https://mlsanity.com/paper/2201.08264/) (2022)
- [Object Relational Graph with Teacher-Recommended Learning for Video Captioning](https://mlsanity.com/paper/2002.11566/) (2020)
- [VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding](https://mlsanity.com/paper/2105.09996/) (2021)
- [Central Similarity Quantization for Efficient Image and Video Retrieval](https://mlsanity.com/paper/1908.00347/) (2019)
- [Camera Calibration and Player Localization in SoccerNet-v2 and Investigation of their Representations for Action Spotting](https://mlsanity.com/paper/2104.09333/) (2021)
- [VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs](https://mlsanity.com/paper/2101.12059/) (2021)
- [Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning](https://mlsanity.com/paper/2003.05162/) (2020)
- [WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types](https://mlsanity.com/paper/2204.06347/) (2022)
- [Hierarchical Modular Network for Video Captioning](https://mlsanity.com/paper/2111.12476/) (2021)
- [Going for GOAL: A Resource for Grounded Football Commentaries](https://mlsanity.com/paper/2211.04534/) (2022)

## Built on by (most cited first, 28 of 28)

- [SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries](https://mlsanity.com/paper/2304.04565/) (2023)
- [MatchTime: Towards Automatic Soccer Game Commentary Generation](https://mlsanity.com/paper/2406.18530/) (2024)
- [Towards Universal Soccer Video Understanding](https://mlsanity.com/paper/2412.01820/) (2024)
- [Multi-Agent System for Comprehensive Soccer Understanding](https://mlsanity.com/paper/2505.03735/) (2025)
- [SIV-Bench: A Video Benchmark for Social Interaction Understanding and Reasoning](https://mlsanity.com/paper/2506.05425/) (2025)
- [SoccerNet-Echoes: A Soccer Game Audio Commentary Dataset](https://mlsanity.com/paper/2405.07354/) (2024)
- [A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision](https://mlsanity.com/paper/2307.03353/) (2023)
- [Language and Multimodal Models in Sports: A Survey of Datasets and Applications](https://mlsanity.com/paper/2406.12252/) (2024)
- [SoccerMaster: A Vision Foundation Model for Soccer Understanding](https://mlsanity.com/paper/2512.11016/) (2025)
- [HOTVCOM: Generating Buzzworthy Comments for Videos](https://mlsanity.com/paper/2409.15196/) (2024)
- [Player-Centric Multimodal Prompt Generation for Large Language Model Based Identity-Aware Basketball Video Captioning](https://mlsanity.com/paper/2507.20163/) (2025)
- [From Multimodal Perception to Strategic Reasoning: A Survey on AI-Generated Game Commentary](https://mlsanity.com/paper/2506.17294/) (2025)
- [Knowledge Guided Entity-aware Video Captioning and A Basketball Benchmark](https://mlsanity.com/paper/2401.13888/) (2024)
- [ExpStar: Towards Automatic Commentary Generation for Multi-discipline Scientific Experiments](https://mlsanity.com/paper/2507.09693/) (2025)
- [Shot2Tactic-Caption: Multi-Scale Captioning of Badminton Videos for Tactical Understanding](https://mlsanity.com/paper/2510.14617/) (2025)
- [TennisExpert: Towards Expert-Level Analytical Sports Video Understanding](https://mlsanity.com/paper/2603.13397/) (2026)
- [BFMD: A Full-Match Badminton Dense Dataset for Dense Shot Captioning](https://mlsanity.com/paper/2603.25533/) (2026)
- [Towards Automatic Soccer Commentary Generation with Knowledge-Enhanced Visual Reasoning](https://mlsanity.com/paper/2604.00057/) (2026)
- [RefereeBench: Are Video MLLMs Ready to be Multi-Sport Referees](https://mlsanity.com/paper/2604.15736/) (2026)
- [SoccerLens: Grounded Soccer Video Understanding Beyond Accuracy](https://mlsanity.com/paper/2605.09598/) (2026)
- [PLayerTV: Advanced Player Tracking and Identification for Automatic Soccer Highlight Clips](https://mlsanity.com/paper/2407.16076/) (2024)
- [Commentary Generation for Soccer Highlights](https://mlsanity.com/paper/2508.07543/) (2025)
- [BoxComm: Benchmarking Category-Aware Commentary Generation and Narration Rhythm in Boxing](https://mlsanity.com/paper/2604.04419/) (2026)
- [BasketEvent: Understanding Who Did What and When in Basketball Videos](https://mlsanity.com/paper/2607.21267/) (2026)
- [NBA_Streaming: A Large-Scale Benchmark for Fine-Grained Basketball Commentary Generation in Continuous Streams](https://mlsanity.com/paper/2608.09200/) (2026)
- [Content Based Video Narration of Gameplay with Vision Language Models](https://mlsanity.com/paper/2608.14016/) (2026)
- [PL-NBA: A Possession-level Universal Basketball Video Dataset Supporting Multiple Visual Understanding Tasks](https://mlsanity.com/paper/2608.19646/) (2026)
- [StreamSoccer: Event-Driven Memory for Streaming Soccer Commentary](https://mlsanity.com/paper/2608.19723/) (2026)

## Abstract (arXiv)

Despite the recent emergence of video captioning models, how to generate vivid, fine-grained video descriptions based on the background knowledge (i.e., long and informative commentary about the domain-specific scenes with appropriate reasoning) is still far from being solved, which however has great applications such as automatic sports narrative. In this paper, we present GOAL, a benchmark of over 8.9k soccer video clips, 22k sentences, and 42k knowledge triples for proposing a challenging new task setting as Knowledge-grounded Video Captioning (KGVC). Moreover, we conduct experimental adaption of existing methods to show the difficulty and potential directions for solving this valuable and applicable task. Our data and code are available at https://github.com/THU-KEG/goal.