# MatchTime: Towards Automatic Soccer Game Commentary Generation

arXiv: https://arxiv.org/abs/2406.18530 · HTML: https://mlsanity.com/paper/2406.18530/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 49 references that are in the map.
- First posted on arXiv 26 Jun 2024.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [Towards Automatic Learning of Procedures from Web Instructional Videos](https://mlsanity.com/paper/1703.09788/) (2017)
- [InternVideo: General Video Foundation Models via Generative and Discriminative Learning](https://mlsanity.com/paper/2212.03191/) (2022)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning](https://mlsanity.com/paper/2302.14115/) (2023)
- [FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding](https://mlsanity.com/paper/2004.06704/) (2020)
- [WikiHow: A Large Scale Text Summarization Dataset](https://mlsanity.com/paper/1810.09305/) (2018)
- [SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos](https://mlsanity.com/paper/1804.04527/) (2018)
- [SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos](https://mlsanity.com/paper/2011.13367/) (2020)
- [FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment](https://mlsanity.com/paper/2204.03646/) (2022)
- [Learning To Recognize Procedural Activities with Distant Supervision](https://mlsanity.com/paper/2201.10990/) (2022)
- [Temporal Alignment Networks for Long-term Video](https://mlsanity.com/paper/2204.02968/) (2022)
- [SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos](https://mlsanity.com/paper/2204.06918/) (2022)
- [SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries](https://mlsanity.com/paper/2304.04565/) (2023)
- [AutoAD II: The Sequel -- Who, When, and What in Movie Audio Description](https://mlsanity.com/paper/2310.06838/) (2023)
- [TacticAI: an AI assistant for football tactics](https://mlsanity.com/paper/2310.10553/) (2023)
- [VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views](https://mlsanity.com/paper/2304.04617/) (2023)
- [AutoAD III: The Prequel -- Back to the Pixels](https://mlsanity.com/paper/2404.14412/) (2024)
- [Temporally-Aware Feature Pooling for Action Spotting in Soccer Broadcasts](https://mlsanity.com/paper/2104.06779/) (2021)
- [GOAL: A Challenging Knowledge-grounded Video Captioning Benchmark for Real-time Soccer Commentary Generation](https://mlsanity.com/paper/2303.14655/) (2023)
- [Learning to Ground Instructional Articles in Videos through Narrations](https://mlsanity.com/paper/2306.03802/) (2023)

## Built on by (most cited first, 30 of 37)

- [LamRA: Large Multimodal Model as Your Advanced Retrieval Assistant](https://mlsanity.com/paper/2412.01720/) (2024)
- [StreamMind: Unlocking Full Frame Rate Streaming Video Dialogue through Event-Gated Cognition](https://mlsanity.com/paper/2503.06220/) (2025)
- [SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence](https://mlsanity.com/paper/2505.17012/) (2025)
- [Towards Universal Soccer Video Understanding](https://mlsanity.com/paper/2412.01820/) (2024)
- [Multi-Agent System for Comprehensive Soccer Understanding](https://mlsanity.com/paper/2505.03735/) (2025)
- [TimeSoccer: An End-to-End Multimodal Large Language Model for Soccer Commentary Generation](https://mlsanity.com/paper/2504.17365/) (2025)
- [SoccerMaster: A Vision Foundation Model for Soccer Understanding](https://mlsanity.com/paper/2512.11016/) (2025)
- [LiveCC: Learning Video LLM with Streaming Speech Transcription at Scale](https://mlsanity.com/paper/2504.16030/) (2025)
- [Domain Adaptation of VLM for Soccer Video Understanding](https://mlsanity.com/paper/2505.13860/) (2025)
- [Player-Centric Multimodal Prompt Generation for Large Language Model Based Identity-Aware Basketball Video Captioning](https://mlsanity.com/paper/2507.20163/) (2025)
- [From Multimodal Perception to Strategic Reasoning: A Survey on AI-Generated Game Commentary](https://mlsanity.com/paper/2506.17294/) (2025)
- [SV3.3B: A Sports Video Understanding Model for Action Recognition](https://mlsanity.com/paper/2507.17844/) (2025)
- [X-Ego: Acquiring Team-Level Tactical Situational Awareness via Cross-Egocentric Contrastive Video Representation Learning](https://mlsanity.com/paper/2510.19150/) (2025)
- [GenTac: Generative Modeling and Forecasting of Soccer Tactics](https://mlsanity.com/paper/2604.11786/) (2026)
- [ExpStar: Towards Automatic Commentary Generation for Multi-discipline Scientific Experiments](https://mlsanity.com/paper/2507.09693/) (2025)
- [WorldCupArena: Fine-Grained Evaluation of Language Models and Deep-Research Agents on Football Forecasting](https://mlsanity.com/paper/2607.18084/) (2026)
- [Do We Need Large VLMs for Spotting Soccer Actions?](https://mlsanity.com/paper/2506.17144/) (2025)
- [Multi-Modal Soccer Scene Analysis with Masked Pre-Training](https://mlsanity.com/paper/2512.19528/) (2025)
- [TennisExpert: Towards Expert-Level Analytical Sports Video Understanding](https://mlsanity.com/paper/2603.13397/) (2026)
- [BFMD: A Full-Match Badminton Dense Dataset for Dense Shot Captioning](https://mlsanity.com/paper/2603.25533/) (2026)
- [Towards Automatic Soccer Commentary Generation with Knowledge-Enhanced Visual Reasoning](https://mlsanity.com/paper/2604.00057/) (2026)
- [Towards Temporal Compositional Reasoning in Long-Form Sports Videos](https://mlsanity.com/paper/2604.22226/) (2026)
- [SoccerLens: Grounded Soccer Video Understanding Beyond Accuracy](https://mlsanity.com/paper/2605.09598/) (2026)
- [SoccerNet 2026 Challenges Results](https://mlsanity.com/paper/2607.07320/) (2026)
- [Towards Comprehensive Basketball Understanding](https://mlsanity.com/paper/2608.23435/) (2026)
- [Commentary Generation for Soccer Highlights](https://mlsanity.com/paper/2508.07543/) (2025)
- [Where is the multimodal goal post? On the Ability of Foundation Models to Recognize Highlights in Videos](https://mlsanity.com/paper/2601.16333/) (2026)
- [Generalizing Sports Feedback Generation by Watching Competitions and Reading Books: A Rock Climbing Case Study](https://mlsanity.com/paper/2602.08996/) (2026)
- [BoxComm: Benchmarking Category-Aware Commentary Generation and Narration Rhythm in Boxing](https://mlsanity.com/paper/2604.04419/) (2026)
- [SoccerRef-Agents: Multi-Agent System for Automated Soccer Refereeing](https://mlsanity.com/paper/2604.23392/) (2026)

## Abstract (arXiv)

Soccer is a globally popular sport with a vast audience, in this paper, we consider constructing an automatic soccer game commentary model to improve the audiences' viewing experience. In general, we make the following contributions: First, observing the prevalent video-text misalignment in existing datasets, we manually annotate timestamps for 49 matches, establishing a more robust benchmark for soccer game commentary generation, termed as SN-Caption-test-align; Second, we propose a multi-modal temporal alignment pipeline to automatically correct and filter the existing dataset at scale, creating a higher-quality soccer game commentary dataset for training, denoted as MatchTime; Third, based on our curated dataset, we train an automatic commentary generation model, named MatchVoice. Extensive experiments and ablation studies have demonstrated the effectiveness of our alignment pipeline, and training model on the curated dataset achieves state-of-the-art performance for commentary generation, showcasing that better alignment can lead to significant performance improvements in downstream tasks.