mlsanityData to 7 Oct 2026

Deep soccer captioning with transformer: dataset, semantics-related losses, and multi-level evaluation

arXiv 2202.05728 ↗First posted 11 Feb 2022cs.CV cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

This work aims at generating captions for soccer videos using deep learning. In this context, this paper introduces a dataset, model, and triple-level evaluation. The dataset consists of 22k caption-clip pairs and three visual features (images, optical flow, inpainting) for ~500 hours of SoccerNet videos. The model is divided into three parts: a transformer learns language, ConvNets learn vision, and a fusion of linguistic and visual features generates captions. The paper suggests evaluating generated captions at three levels: syntax (the commonly used evaluation metrics such as BLEU-score and CIDEr), meaning (the quality of descriptions for a domain expert), and corpus (the diversity of generated captions). The paper shows that the diversity of generated captions has improved (from 0.07 reaching 0.18) with semantics-related losses that prioritize selected words. Semantics-related losses and the utilization of more visual features (optical flow, inpainting) improved the normalized captioning score by 28%. The web page of this work: https://sites.google.com/view/soccercaptioninghttps://sites.google.com/view/soccercaptioning

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 20 of its 30 references that are in the map. First posted on arXiv 11 Feb 2022.

Builds on

20 of 30 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  3. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  4. 2020SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videoscited by 129
  5. 2020Multi-modal Dense Video Captioningcited by 113
  6. 2020Spatio-Temporal Graph for Video Captioning with Knowledge Distillationcited by 109
  7. 2014Translating Videos to Natural Language Using Deep Recurrent Neural Networkscited by 95
  8. 2015Describing Videos by Exploiting Temporal Structurecited by 88
  9. 2018Grounded Video Descriptioncited by 78
  10. 2019Compositional Generalization in Image Captioningcited by 23
  11. 2020Improved Soccer Action Spotting using both Audio and Video Streamscited by 21
  12. 2021RMS-Net: Regression and Masking for Soccer Event Spottingcited by 19
8 more
  1. 2019A Case Study on Combining ASR and Visual Features for Generating Instructional Video Captionscited by 15
  2. 2017An empirical study on the effectiveness of images in Multimodal Neural Machine Translationcited by 10
  3. 2017Multimodal Compact Bilinear Pooling for Multimodal Neural Machine Translationcited by 8
  4. 2021Multi-level Attention Fusion Network for Audio-visual Event Recognitioncited by 7
  5. 2017Modulating and attending the source image during encoding improves Multimodal Translationcited by 5
  6. 2018UMONS Submission for WMT18 Multimodal Translation Taskcited by 2
  7. 2020AVECL-UMONS database for audio-visual event classification and localizationcited by 2
  8. 2017Visually Grounded Word Embeddings and Richer Visual Features for Improving Multimodal Neural Machine Translationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 2; 2026: 1.

4 papers.

  1. 2023SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentariescited by 57
  2. 2025Player-Centric Multimodal Prompt Generation for Large Language Model Based Identity-Aware Basketball Video Captioningcited by 7
  3. 2025Shot2Tactic-Caption: Multi-Scale Captioning of Badminton Videos for Tactical Understandingcited by 2
  4. 2026Sentinel2Cap: A Human-Annotated Benchmark Dataset for Multimodal Remote Sensing Image Captioningcited by 0